[HIVE-1409] File format information is retrieved from first partition

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.0
• Fix Version/s: 0.6.0
• Component/s: Query Processor
• Hadoop Flags: Reviewed
• Assignee: Paul Yang
• Reporter: Paul Yang
• Created: 15/Jun/10 03:26
• Updated: 17/Dec/11 00:03
• Resolved: 15/Jun/10 22:04

Description

Currently, if no partitions match the partition predicate, the first partition is used to retrieve the file format. This can cause an problem if the table is set to use RCFile, but the first partition uses SequenceFile:

java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.hive.ql.io.RCFile$KeyBuffer.()
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
	at org.apache.hadoop.mapred.SequenceFileRecordReader.createKey(SequenceFileRecordReader.java:65)
	at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.createKey(CombineHiveRecordReader.java:76)
	at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.createKey(CombineHiveRecordReader.java:42)
	at org.apache.hadoop.hive.shims.Hadoop20Shims$CombineFileRecordReader.createKey(Hadoop20Shims.java:212)
	at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.createKey(MapTask.java:167)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
	at org.apache.hadoop.mapred.Child.main(Child.java:159)
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.ql.io.RCFile$KeyBuffer.()
	at java.lang.Class.getConstructor0(Class.java:2706)
	at java.lang.Class.getDeclaredConstructor(Class.java:1985)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
	... 9 more

The proposed change is to use the table's metadata in such cases.