Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-1646

[Python] pyarrow.array cannot handle NumPy scalar types

    Details

      Description

      Example repro

      In [1]: import pyarrow as pa
      impo
      In [2]: import numpy as np
      
      In [3]: pa.array([np.random.randint(0, 10, size=5), None])
      ---------------------------------------------------------------------------
      ArrowInvalid                              Traceback (most recent call last)
      <ipython-input-3-b364fa5d75aa> in <module>()
      ----> 1 pa.array([np.random.randint(0, 10, size=5), None])
      
      /home/wesm/code/arrow/python/pyarrow/array.pxi in pyarrow.lib.array (/home/wesm/code/arrow/python/build/temp.linux-x86_64-3.5/lib.cxx:24892)()
          171         if mask is not None:
          172             raise ValueError("Masks only supported with ndarray-like inputs")
      --> 173         return _sequence_to_array(obj, size, type, pool)
          174 
          175 
      
      /home/wesm/code/arrow/python/pyarrow/array.pxi in pyarrow.lib._sequence_to_array (/home/wesm/code/arrow/python/build/temp.linux-x86_64-3.5/lib.cxx:23496)()
           23     if type is None:
           24         with nogil:
      ---> 25             check_status(ConvertPySequence(sequence, pool, &out))
           26     else:
           27         if size is None:
      
      /home/wesm/code/arrow/python/pyarrow/error.pxi in pyarrow.lib.check_status (/home/wesm/code/arrow/python/build/temp.linux-x86_64-3.5/lib.cxx:7876)()
           75         message = frombytes(status.message())
           76         if status.IsInvalid():
      ---> 77             raise ArrowInvalid(message)
           78         elif status.IsIOError():
           79             raise ArrowIOError(message)
      
      ArrowInvalid: /home/wesm/code/arrow/cpp/src/arrow/python/builtin_convert.cc:740 code: InferArrowTypeAndSize(obj, &size, &type)
      /home/wesm/code/arrow/cpp/src/arrow/python/builtin_convert.cc:319 code: InferArrowType(obj, out_type)
      /home/wesm/code/arrow/cpp/src/arrow/python/builtin_convert.cc:299 code: seq_visitor.Visit(obj)
      /home/wesm/code/arrow/cpp/src/arrow/python/builtin_convert.cc:180 code: VisitElem(ref, level)
      Error inferring Arrow data type for collection of Python objects. Got Python object of type ndarray but can only handle these types: bool, float, integer, date, datetime, bytes, unicode
      

      If these inner values are converted to Python built-in int types then it works fine

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                xhochy Uwe L. Korn
                Reporter:
                wesmckinn Wes McKinney
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: