[ARROW-10684] [Rust] Logical equality should consider parent array nullability

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 3.0.0
• Component/s: Rust
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/26635
• Assignee: Neville Dipale
• Reporter: Neville Dipale
• Created: 22/Nov/20 10:15
• Updated: 11/Jan/23 08:14
• Resolved: 29/Nov/20 12:34

Description

When creating a struct array with a primitive child array, it is possible for the child to be non-nullable, while its parent struct array is nullable.

In this scenario, the child array's slots where the parent is null, become invalidated, such that an array with [1, 2, 3] having slot 2 being null, should be interpreted as [1, 0, 3].

This issue becomes evident in Parquet roundtrip tests, as we end up not able to correctly compare nested structures that have non-null children.

The specification caters for the above behaviour, see http://arrow.apache.org/docs/format/Columnar.html#struct-layout .

When a struct has nulls, its child array(s) nullability is subject to the parent struct.

Issue Links

• links to: GitHub Pull Request #8739