[ARROW-11632] [Rust] csv::Reader doesn't propagate schema metadata to RecordBatches

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 3.0.1, 4.0.0
• Component/s: Rust
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/27495
• Assignee: Unassigned
• Reporter: Marko Mikulicic
• Created: 15/Feb/21 17:27
• Updated: 11/Jan/23 08:21
• Resolved: 16/Feb/21 19:52

Description

Rust csv::Reader doesn't propagate schema metadata to RecordBatches it creates:

 

let reader = csv::ReaderBuilder::new()
    .with_schema(Arc::new(Schema::new_with_metadata(fields, metadata)))
    .build(file)
    .unwrap();

let batch = reader.next().unwrap().unwrap();

assert_eq!(reader.schema(), batch.schema());

 

The parquet writer crate complains about the schema non matching with this error:
Record batch schema does not match writer schema
 

 

Issue Links

• links to: GitHub Pull Request #9501