Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2124

[Python] ArrowInvalid raised if the first item of a nested list of numpy arrays is empty

    XMLWordPrintableJSON

Details

    Description

      See example below:

      In [1]: import numpy as np
      
      In [2]: import pandas as pd
      
      In [3]: import pyarrow as pa
      
      In [4]: num_lists = [[2,3,4], [3,6,7,8], [], [2]]
      
      In [5]: series = pd.Series([np.array(s, dtype=float) for s in num_lists])
      
      In [6]: pa.array(series)
      Out[6]: 
      <pyarrow.lib.ListArray object at 0x7f0db8ad1688>
      [
        [2.0,
         3.0,
         4.0],
        [3.0,
         6.0,
         7.0,
         8.0],
        [],
        [2.0]
      ]
      
      In [7]: num_lists.append([])
      
      In [8]: series = pd.Series([np.array(s, dtype=float) for s in num_lists])
      
      In [9]: pa.array(series)
      Out[9]: 
      <pyarrow.lib.ListArray object at 0x7f0db8ad1e58>
      [
        [2.0,
         3.0,
         4.0],
        [3.0,
         6.0,
         7.0,
         8.0],
        [],
        [2.0],
        []
      ]
      
      In [10]: num_lists.insert(0, [])
      
      In [11]: series = pd.Series([np.array(s, dtype=float) for s in num_lists])
      
      In [12]: pa.array(series)
      ---------------------------------------------------------------------------
      ArrowInvalid                              Traceback (most recent call last)
      <ipython-input-99-fc3a903278e6> in <module>()
      ----> 1 pa.array(series)
      
      array.pxi in pyarrow.lib.array()
      
      array.pxi in pyarrow.lib._ndarray_to_array()
      
      error.pxi in pyarrow.lib.check_status()
      
      ArrowInvalid: trying to convert NumPy type object but got float64
      

      Attachments

        Issue Links

          Activity

            People

              uwe Uwe Korn
              gsakkis George Sakkis
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: