[ARROW-10261] [Rust] [BREAKING] Lists should take Field instead of DataType

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.1
• Fix Version/s: 3.0.0
• Component/s: Integration, Rust
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/26257
• Assignee: Neville Dipale
• Reporter: Neville Dipale
• Created: 10/Oct/20 01:31
• Updated: 11/Jan/23 08:12
• Resolved: 07/Nov/20 12:18

Description

There is currently no way of tracking nested field metadata on lists. For example, if a list's children are nullable, there's no way of telling just by looking at the Field.

This causes problems with integration testing, and also affects Parquet roundtrips.

I propose the breaking change of [Large|FixedSize]List taking a Field instead of Box<DataType>, as this will overcome this issue, and ensure that the Rust implementation passes integration tests.

CC andygrove jorgecarleitao alamb  jhorstmann (carols10cents as this addresses some of the roundtrip failures).

I'm leaning towards this landing in 3.0.0, as I'd love for us to have completed or made significant traction on the Arrow Parquet writer (and reader), and integration testing, by then.

Issue Links

• links to: GitHub Pull Request #8608