[DRILL-5292] Better Parquet handling of sparse columns

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.10.0
• Component/s: Storage - Parquet
• Assignee: Unassigned
• Reporter: Nate Putnam
• Created: 22/Feb/17 22:22
• Updated: 24/Feb/17 17:38

Description

It appears the current implantation of ParquetRecordReader will fill in missing columns between files as a NullableIntVector. It would be better if the code could determine if that column was defined in a different file (and didn't conflict) and use the defined data type.