[ARROW-7273] [Python] Non-nullable null field is allowed / crashes when writing to parquet

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: C++, Python
• Labels: parquet pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/23562
• Assignee: Wes McKinney
• Reporter: Joris Van den Bossche
• Created: 28/Nov/19 13:12
• Updated: 11/Jan/23 07:52
• Resolved: 28/Jun/20 14:41

Description

It seems to be possible to create a "non-nullable null field". While this does not make any sense (so already a reason to disallow this I think), this can also lead to crashed in further operations, such as writing to parquet:

In [18]: table = pa.table([pa.array([None, None], pa.null())], schema=pa.schema([pa.field('a', pa.null(), nullable=False)]))

In [19]: table
Out[19]:
pyarrow.Table
a: null not null

In [20]: pq.write_table(table, "test_null.parquet")
WARNING: Logging before InitGoogleLogging() is written to STDERR
F1128 14:08:30.267439 27560 column_writer.cc:837]  Check failed: (nullptr) != (values)
*** Check failure stack trace: ***
Aborted (core dumped)

Issue Links

• links to: GitHub Pull Request #7562