Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.11.0
-
None
-
None
-
Reviewed
Description
Steps to reproduce:
$ git clone https://github.com/apache/hive.git hive-profiler-npe
Initialized empty Git repository in /home/brock/hive-profiler-npe/.git/
remote: Counting objects: 73654, done.
remote: Compressing objects: 100% (15383/15383), done.
remote: Total 73654 (delta 44331), reused 71338 (delta 43054)
Receiving objects: 100% (73654/73654), 42.78 MiB | 1.69 MiB/s, done.
Resolving deltas: 100% (44331/44331), done.
$ cd hive-profiler-npe/
$ ant clean package
$ cd build/dist/
$ ./bin/hive
hive> DROP TABLE IF EXISTS users;
hive> CREATE TABLE users (
user string,
passwd string,
uid int,
gid int,
name string,
home string,
shell string
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ':'
STORED AS TEXTFILE;
hive> LOAD DATA LOCAL INPATH '/etc/passwd' INTO TABLE users;
hive> set hive.exec.operator.hooks=org.apache.hadoop.hive.ql.profiler.HiveProfiler;
set hive.exec.operator.hooks=org.apache.hadoop.hive.ql.profiler.HiveProfiler
hive> set hive.exec.post.hooks=org.apache.hadoop.hive.ql.hooks.HiveProfilerResultsHook;
set hive.exec.post.hooks=org.apache.hadoop.hive.ql.hooks.HiveProfilerResultsHook
hive> SET hive.exec.mode.local.auto=false;
SET hive.exec.mode.local.auto=false
hive> SET hive.task.progress=true;
SET hive.task.progress=true
hive>
> select count(1) from users;
select count(1) from users
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302131617_0022, Tracking URL = http://localhost.localdomain:50030/jobdetails.jsp?jobid=job_201302131617_0022
Kill Command = /usr/local/hadoop-1.0.4/libexec/../bin/hadoop job -kill job_201302131617_0022
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-16 10:24:14,215 Stage-1 map = 0%, reduce = 0%
2013-02-16 10:24:44,354 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201302131617_0022 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://localhost.localdomain:50030/jobdetails.jsp?jobid=job_201302131617_0022
Examining task ID: task_201302131617_0022_m_000002 (and more) from job job_201302131617_0022
Task with the most failures(4):
-----
Task ID:
task_201302131617_0022_m_000000
URL:
http://localhost.localdomain:50030/taskdetails.jsp?jobid=job_201302131617_0022&tipid=task_201302131617_0022_m_000000
-----
Diagnostic Messages for this Task:
java.lang.RuntimeException: Hive Runtime Error while closing operators
at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:227)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.profiler.HiveProfilePublisher.publishStat(HiveProfilePublisher.java:85)
at org.apache.hadoop.hive.ql.profiler.HiveProfiler.close(HiveProfiler.java:110)
at org.apache.hadoop.hive.ql.exec.Operator.closeOperatorHooks(Operator.java:452)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:605)
at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:194)
... 8 more
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 Reduce: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
