[ARROW-9577] [Python][C++] posix_madvise error on Debian in pyarrow 1.0.0

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.0.1, 2.0.0
• Component/s: Python
• Labels: pull-request-available
• Environment: Installed with Miniconda (for Debian; used pip for the Ubuntu test)
• External issue URL: https://github.com/apache/arrow/issues/25642
• Assignee: Antoine Pitrou
• Reporter: Jim Pivarski
• Created: 27/Jul/20 23:24
• Updated: 11/Jan/23 08:07
• Resolved: 12/Aug/20 13:16

Description

The following writes and reads back from a Parquet file in both pyarrow 0.17.0 and 1.0.0 on Ubuntu 18.04:
 

>>> import pyarrow.parquet
>>> a = pyarrow.array([[1.1, 2.2, 3.3], [], [4.4, 5.5]])
>>> t = pyarrow.Table.from_batches([pyarrow.RecordBatch.from_arrays([a], ["stuff"])])
>>> pyarrow.parquet.write_table(t, "stuff.parquet")
>>> t2 = pyarrow.parquet.read_table("stuff.parquet") 

 
However, the same thing raises the following exception on Debian 9 (stretch) in pyarrow 1.0.0 but not in pyarrow 0.17.0:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jpivarski/miniconda3/lib/python3.7/site-packages/pyarrow/parquet.py", line 1564, in read_table
    filters=filters,
  File "/home/jpivarski/miniconda3/lib/python3.7/site-packages/pyarrow/parquet.py", line 1433, in __init__
    partitioning=partitioning)
  File "/home/jpivarski/miniconda3/lib/python3.7/site-packages/pyarrow/dataset.py", line 667, in dataset
    return _filesystem_dataset(source, **kwargs)
  File "/home/jpivarski/miniconda3/lib/python3.7/site-packages/pyarrow/dataset.py", line 434, in _filesystem_dataset
    return factory.finish(schema)
  File "pyarrow/_dataset.pyx", line 1451, in pyarrow._dataset.DatasetFactory.finish
  File "pyarrow/error.pxi", line 122, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
OSError: posix_madvise failed. Detail: [errno 0] Success

It's a little odd that the error says that it failed with "detail: success". That suggests to me that an "if" predicate is backward (missing "not"), which might only be triggered on some OS/distributions.

Issue Links

• links to: GitHub Pull Request #7904