Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
Reproduced on Cloudera 3
Description
When a block is corrupt or missing from HDFS (due to the failure of multiple nodes), Hive jobs hang mysteriously at the beginning (without starting the M/R job):
Launching Job 2 out of 4 Number of reduce tasks not specified. Estimated from input data size: 2 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant number of reducers: set mapred.reduce.tasks=<number>
The java process takes 100% of 1 CPU, and never stops, i had to kill manually all my hive jobs.
It would be nice if the processes could fail with an appropriate exception rather than hanging.