Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-14320

[Python] List array conversion to Numpy N-d array

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 5.0.0
    • None
    • Python
    • None

    Description

      When converting a single-dimensional array to numpy, the dtype is preserved

      import pyarrow as pa
      x = pa.array([.234,.345,.456])
      x.to_numpy().dtype # dtype('float64')

      But when doing the same for a multi-dimensional array, the dtype is lost and cannot be set manually

      x = pa.array([[1,2,3],[4,5,6]]).to_numpy(zero_copy_only=False)
      print(x.dtpye) # object
      x.astype(np.float64) # ValueError: setting an array element with a sequence.

      Which is to say that numpy believes this array is not uniform. The only way to get it to the proper dtype is to convert it to a python list then back to a numpy array.

      Is there another way to achieve this? Or, at least, can it be fixed such that we can manually set the dtype of the numpy array after conversion?

      I know that pyarrow doesn't support ndarrays with ndim>1 (https://issues.apache.org/jira/browse/ARROW-5645) but I was curious if this can be achieved going the other way.

      Attachments

        Activity

          People

            Unassigned Unassigned
            benepstein1 Ben Epstein
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: