-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 0.14.0, 1.0.0, 1.1.0, 1.2.0, 1.2.1
-
Fix Version/s: 2.0.0
-
Component/s: Serializers/Deserializers
-
Labels:None
-
Target Version/s:
If a zero length file is in the top level directory housing an external avro table, all hive queries on the table fail.
This issue is that org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader creates a new org.apache.avro.file.DataFileReader and DataFileReader throws an exception when trying to read an empty file (because the empty file lacks the magic number marking it as avro).
AvroGenericRecordReader should detect an empty file and then behave reasonably.
Caused by: java.io.IOException: Not a data file.
at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:102)
at org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97)
at org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader.<init>(AvroGenericRecordReader.java:81)
at org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat.getRecordReader(AvroContainerInputFormat.java:51)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:246)
... 25 more
- duplicates
-
HIVE-7316 Hive fails on zero length files
-
- Resolved
-
- relates to
-
HIVE-12607 Hive fails on zero length sequence files
-
- Open
-
-
AVRO-1530 Java DataFileStream does not allow distinguishing between empty files and corrupt files
-
- Resolved
-