[ARROW-15478] [C++] Creating (or casting to) list array with non-nullable field doesn't check nulls

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: C++
• External issue URL: https://github.com/apache/arrow/issues/30954
• Assignee: Unassigned
• Reporter: Joris Van den Bossche
• Created: 27/Jan/22 15:00
• Updated: 11/Jan/23 11:37

Description

When creating a ListArray where you indicate that the values field is not nullable, you can actually create the array with nulls without this is being validated:

>>> typ = pa.list_(pa.field("element", pa.int64(), nullable=False))
>>> arr = pa.array([[1, 2], [3, 4, None]], typ)
>>> arr
<pyarrow.lib.ListArray object at 0x7f75bdeba760>
[
  [
    1,
    2
  ],
  [
    3,
    4,
    null
  ]
]

>>> arr.type
ListType(list<element: int64 not null>)

Also explicitly validating it doesn't raise:

>>> arr.validate(full=True)

Is this something we should check?
What guarantees do we attach to the nullability of a field of a nested type?

Issue Links

• relates to: ARROW-16651 [Python] Casting Table to new schema ignores nullability of fields