[HIVE-13632] Hive failing on insert empty array into parquet table

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.3.0, 2.1.0
• Component/s: Serializers/Deserializers
• Assignee: Yongzhi Chen
• Reporter: Yongzhi Chen
• Created: 27/Apr/16 23:20
• Updated: 18/Feb/22 03:07
• Resolved: 05/May/16 15:10

Description

The insert will fail with following stack:

by: parquet.io.ParquetEncodingException: empty fields are illegal, the field should be ommited completely instead
	at parquet.io.MessageColumnIO$MessageColumnIORecordConsumer.endField(MessageColumnIO.java:271)
	at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$ListDataWriter.write(DataWritableWriter.java:271)
	at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$GroupDataWriter.write(DataWritableWriter.java:199)
	at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$MessageDataWriter.write(DataWritableWriter.java:215)
	at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter.write(DataWritableWriter.java:88)
	at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:59)
	at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:31)
	at parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:116)
	at parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:123)
	at parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:42)
	at org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:111)
	at org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:124)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:697)

Reproduce:

create table test_small (
key string,
arrayValues array<string>)
stored as parquet;
insert into table test_small select 'abcd', array() from src limit 1;

Issue Links

• is related to: PARQUET-596 Parquet need to support empty list and empty map