[ARROW-8122] [Python] Empty numpy arrays with shape cannot be deserialized

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.16.0
• Fix Version/s: 0.17.0
• Component/s: Python
• Labels: pull-request-available serialization
• External issue URL: https://github.com/apache/arrow/issues/24331
• Assignee: Wenjun Si
• Reporter: Wenjun Si
• Created: 15/Mar/20 01:18
• Updated: 11/Jan/23 07:58
• Resolved: 19/Mar/20 00:49

Description

In PyArrow 0.16.0, when we try to deserialize a serialized empty Numpy Array with shape, for instance, np.array([[], []]), an ArrowInvalid is raised.

Code reproducing this error:

import numpy as np
import pyarrow
arr = np.array([[], []])
pyarrow.deserialize(pyarrow.serialize(arr).to_buffer())  # this line cannot work

and the error stack is

Traceback (most recent call last):
  File "/Users/wenjun/miniconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3326, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-4-0ace9226dd72>", line 1, in <module>
    pyarrow.deserialize(pyarrow.serialize(arr).to_buffer())
  File "pyarrow/serialization.pxi", line 476, in pyarrow.lib.deserialize
  File "pyarrow/serialization.pxi", line 438, in pyarrow.lib.deserialize_from
  File "pyarrow/serialization.pxi", line 414, in pyarrow.lib.read_serialized
  File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: strides must not involve buffer over run

The same code works in PyArrow 0.15.x

Issue Links

• relates to: ARROW-7996 [Python] Error serializing empty pandas DataFrame with pyarrow
• links to: GitHub Pull Request #6624