Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-5857

[Python] converting multidimensional numpy arrays to nested list type

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Python
    • Labels:
      None

      Description

      Currently we only support 1-dimensional numpy arrays:

      In [28]: pa.array([np.array([[1, 2], [3, 4]])], type=pa.list_(pa.list_(pa.int64()))) 
      ...
      ArrowInvalid: Can only convert 1-dimensional array values
      

      So to create a nested list array, you can do that with lists of lists or object numpy arrays with arrays as elements. We could expand this support to multi-dimensional numpy arrays.

      I am not sure we should do inference by default for this case, but at least when specifying a nested ListType, this would be nice.

      It can be an alternative way to have some support for tensors, next to an ExtensionType (ARROW-1614 / ARROW-5819)

      Related discussions: https://lists.apache.org/thread.html/9b142c1709aa37dc35f1ce8db4e1ced94fcc4cdd96cc72b5772b373b@%3Cdev.arrow.apache.org%3E, https://github.com/apache/arrow/issues/4802

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                jorisvandenbossche Joris Van den Bossche
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: