[ARROW-5857] [Python] converting multidimensional numpy arrays to nested list type

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/22274
• Assignee: Unassigned
• Reporter: Joris Van den Bossche
• Created: 04/Jul/19 21:32
• Updated: 11/Jan/23 07:43
• Resolved: 09/Jul/19 16:26

Description

Currently we only support 1-dimensional numpy arrays:

In [28]: pa.array([np.array([[1, 2], [3, 4]])], type=pa.list_(pa.list_(pa.int64()))) 
...
ArrowInvalid: Can only convert 1-dimensional array values

So to create a nested list array, you can do that with lists of lists or object numpy arrays with arrays as elements. We could expand this support to multi-dimensional numpy arrays.

I am not sure we should do inference by default for this case, but at least when specifying a nested ListType, this would be nice.

It can be an alternative way to have some support for tensors, next to an ExtensionType (ARROW-1614 / ARROW-5819)

Related discussions: https://lists.apache.org/thread.html/9b142c1709aa37dc35f1ce8db4e1ced94fcc4cdd96cc72b5772b373b@%3Cdev.arrow.apache.org%3E, https://github.com/apache/arrow/issues/4802

Issue Links

• is duplicated by: ARROW-5645 [Python] Support inferring nested ndarray with ndim > 1