[ARROW-2124] [Python] ArrowInvalid raised if the first item of a nested list of numpy arrays is empty

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.9.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/18094
• Assignee: Uwe Korn
• Reporter: George Sakkis
• Created: 10/Feb/18 11:09
• Updated: 11/Jan/23 07:19
• Resolved: 12/Feb/18 21:24

Description

See example below:

In [1]: import numpy as np

In [2]: import pandas as pd

In [3]: import pyarrow as pa

In [4]: num_lists = [[2,3,4], [3,6,7,8], [], [2]]

In [5]: series = pd.Series([np.array(s, dtype=float) for s in num_lists])

In [6]: pa.array(series)
Out[6]: 
<pyarrow.lib.ListArray object at 0x7f0db8ad1688>
[
  [2.0,
   3.0,
   4.0],
  [3.0,
   6.0,
   7.0,
   8.0],
  [],
  [2.0]
]

In [7]: num_lists.append([])

In [8]: series = pd.Series([np.array(s, dtype=float) for s in num_lists])

In [9]: pa.array(series)
Out[9]: 
<pyarrow.lib.ListArray object at 0x7f0db8ad1e58>
[
  [2.0,
   3.0,
   4.0],
  [3.0,
   6.0,
   7.0,
   8.0],
  [],
  [2.0],
  []
]

In [10]: num_lists.insert(0, [])

In [11]: series = pd.Series([np.array(s, dtype=float) for s in num_lists])

In [12]: pa.array(series)
---------------------------------------------------------------------------
ArrowInvalid                              Traceback (most recent call last)
<ipython-input-99-fc3a903278e6> in <module>()
----> 1 pa.array(series)

array.pxi in pyarrow.lib.array()

array.pxi in pyarrow.lib._ndarray_to_array()

error.pxi in pyarrow.lib.check_status()

ArrowInvalid: trying to convert NumPy type object but got float64

Issue Links

• links to: GitHub Pull Request #1586