[ARROW-10656] [Rust] New RecordBatch requires exact match of Data Types

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Rust
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/18382
• Assignee: Neville Dipale
• Reporter: Christoph Schulze
• Created: 19/Nov/20 16:31
• Updated: 11/Jan/23 08:14
• Resolved: 01/Jan/21 10:47

Description

When instanciating a new RecordBatch with try_new(), the data types of columns are checked to match their corresponding fields in the schema with ==. The == operator will consider all attribues of the two data types and compare strictly if all values are equal. However, a code comment above this comparison indicates [1]:

 list types can have different names, but we only need the data types to be the same

Since ARROW-10261 (PR) was merged lists contain a Field instead of just a DataType. Therefore, the values of this field are striclty compared. This behavior contradicts the comment.

Solution

 

The data type comparison in try_new() should be changed into only matching data types, including the nested ones, but leaving out other values.

 

[1] src/record_batch.rs:103

Issue Links

• links to: GitHub Pull Request #8715
• links to: GitHub Pull Request #8802
• links to: GitHub Pull Request #8988