[ARROW-18411] [Python] MapType comparison ignores nullable flag of item_field

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Component/s: Python
• Environment: pyarrow==10.0.1
• External issue URL: https://github.com/apache/arrow/issues/33574
• Assignee: Will Jones
• Reporter: &res
• Created: 27/Nov/22 14:06
• Updated: 11/Jan/23 12:00
• Resolved: 05/Jan/23 18:04

Description

By default MapType value fields are nullable:

 pa.map_(pa.string(), pa.int32()).item_field.nullable == True 

It is possible to mark the value field of a MapType as not-nullable:

 pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=False)).item_field.nullable == False

But comparing these two types, that are semantically different, returns True:

pa.map_(pa.string(), pa.int32()) == pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=False)) # Returns True 

So it looks like the comparison omits the nullable flag. 

import pyarrow as pa

map_type = pa.map_(pa.string(), pa.int32())
non_null_map_type = pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=False))
nullable_map_type = pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=True))
map_type_different_field_name = pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=True))

assert nullable_map_type == map_type  # Wrong
assert str(nullable_map_type) == str(map_type)
assert str(non_null_map_type) == str(map_type) # Wrong
assert non_null_map_type == map_type
assert non_null_map_type.item_type == map_type.item_type
assert non_null_map_type.item_field != map_type.item_field
assert non_null_map_type.item_field.nullable != map_type.item_field.nullable
assert non_null_map_type.item_field.name == map_type.item_field.name
assert map_type == map_type_different_field_name # This makes sense

Issue Links

• is related to: ARROW-14999 [C++] List types with different field names are not equal