[ARROW-2020] [Python] Parquet segfaults if coercing ns timestamps and writing 96-bit timestamps

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.8.0
• Fix Version/s: 0.10.0
• Component/s: Python
• Labels: timestamps
• Environment: OS: Mac OS X 10.13.2 ; Python: 3.6.4 ; PyArrow: 0.8.0
• External issue URL: https://github.com/apache/arrow/issues/1498
• Assignee: Joshua Storck
• Reporter: Diego Argueta
• Created: 24/Jan/18 02:53
• Updated: 11/Jan/23 07:18
• Resolved: 17/Apr/18 19:21

Description

If you try to write a PyArrow table containing nanosecond-resolution timestamps to Parquet using `coerce_timestamps` and `use_deprecated_int96_timestamps=True`, the Arrow library will segfault.

The crash doesn't happen if you don't coerce the timestamp resolution or if you don't use 96-bit timestamps.

 

 

To Reproduce:

 

 
import datetime

import pyarrow
from pyarrow import parquet

schema = pyarrow.schema([
    pyarrow.field('last_updated', pyarrow.timestamp('ns')),
])

data = [
    pyarrow.array([datetime.datetime.now()], pyarrow.timestamp('ns')),
]

table = pyarrow.Table.from_arrays(data, ['last_updated'])

with open('test_file.parquet', 'wb') as fdesc:
    parquet.write_table(table, fdesc,
                        coerce_timestamps='us',  # 'ms' works too
                        use_deprecated_int96_timestamps=True)

 

See attached file for the crash report.

 

Issue Links

• Blocked: PARQUET-1274 [Python] SegFault in pyarrow.parquet.write_table with specific options