[ARROW-6697] [Rust] [DataFusion] Validate that all parquet partitions have the same schema

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Rust, Rust - DataFusion
• External issue URL: https://github.com/apache/arrow/issues/23043
• Assignee: Andy Grove
• Reporter: Andy Grove
• Created: 25/Sep/19 14:10
• Updated: 11/Jan/23 07:48
• Resolved: 24/Dec/20 18:03

Description

When reading a partitioned Parquet file in DataFusion, the schema is read from the first partition and it is assumed that all other partitions have the same schema.

It would be better to actually validate that all of the partitions have the same schema since there is no support for schema merging yet.