[HIVE-4605] Hive job fails while closing reducer output - Unable to rename

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.11.0, 0.12.0, 0.13.0, 0.13.1, 2.3.0
• Component/s: Query Processor
• Labels: pull-request-available

Environment

OS: 2.6.18-194.el5xen #1 SMP Fri Apr 2 15:34:40 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
Hadoop 1.1.2

• Assignee: Oleksiy Sayankin
• Reporter: Link Qian
• Created: 24/May/13 08:07
• Updated: 01/Mar/22 00:17

Description

1, create a table with ORC storage model

create table iparea_analysis_orc (network int, ip string,  .... )
stored as ORC;

2, insert table iparea_analysis_orc select network, ip, .... , the script success, but failed after add OVERWRITE keyword. the main error log list as here.

java.lang.RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://qa3hop001.uucun.com:9000/tmp/hive-hadoop/hive_2013-05-24_15-11-06_511_7746839019590922068/_task_tmp.-ext-10000/_tmp.000000_0 to: hdfs://qa3hop001.uucun.com:9000/tmp/hive-hadoop/hive_2013-05-24_15-11-06_511_7746839019590922068/_tmp.-ext-10000/000000_0
	at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:317)
	at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:530)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:421)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: hdfs://qa3hop001.uucun.com:9000/tmp/hive-hadoop/hive_2013-05-24_15-11-06_511_7746839019590922068/_task_tmp.-ext-10000/_tmp.000000_0 to: hdfs://qa3hop001.uucun.com:9000/tmp/hive-hadoop/hive_2013-05-24_15-11-06_511_7746839019590922068/_tmp.-ext-10000/000000_0
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:197)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$300(FileSinkOperator.java:108)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:867)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:588)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:597)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:597)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:597)
	at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:309)
	... 7 more

Issue Links

• is cloned by: HIVE-25452 Hive job fails while closing reducer output - Unable to rename
• relates to: HIVE-7273 Hive job fails due to unable to rename reducer output file
• links to: GitHub Pull Request #2909
• links to: Review board