[HIVE-4627] Total ordering of Hive output

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 0.11.0
• Release Note: I'm not able to reproduce this behavior. I now suspect the input data set was empty, causing this runtime exception.
• Assignee: Unassigned
• Reporter: Nick Dimiduk
• Created: 29/May/13 16:58
• Updated: 04/Feb/14 01:35
• Resolved: 04/Feb/14 01:35

Description

I'd like to use Hive to generate HFiles for HBase. I started off by following the instructions on the wiki, but that took me only so far. TotalOrderPartitioning didn't work. That took me to this post which points out that Hive partitions on value instead of key. A patched TOP brings me to this error:

2013-05-17 21:00:47,781 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.RuntimeException: Hive Runtime Error while closing operators: java.io.IOException: No files found in hdfs://ip-10-191-3-134.ec2.internal:8020/tmp/hive-hrt_qa/hive_2013-05-17_20-58-58_357_6896546413926013201/_task_tmp.-ext-10000/_tmp.000000_0
	at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:317)
	at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:532)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:421)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: No files found in hdfs://ip-10-191-3-134.ec2.internal:8020/tmp/hive-hrt_qa/hive_2013-05-17_20-58-58_357_6896546413926013201/_task_tmp.-ext-10000/_tmp.000000_0
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:183)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:865)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:588)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:597)
	at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:309)
	... 7 more
Caused by: java.io.IOException: No files found in hdfs://ip-10-191-3-134.ec2.internal:8020/tmp/hive-hrt_qa/hive_2013-05-17_20-58-58_357_6896546413926013201/_task_tmp.-ext-10000/_tmp.000000_0
	at org.apache.hadoop.hive.hbase.HiveHFileOutputFormat$1.close(HiveHFileOutputFormat.java:142)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:180)
	... 11 more