[ARROW-2774] [Python] Implement pa.array() with type=union type

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/19157
• Assignee: Unassigned
• Reporter: Wes McKinney
• Created: 01/Jul/18 22:18
• Updated: 11/Jan/23 07:23

Description

It would be useful to be able to generate unions during type inference:

In [11]: pa.array([{'a': 1, 'b': 'string'}, {'b': 2}])
---------------------------------------------------------------------------
ArrowTypeError                            Traceback (most recent call last)
<ipython-input-11-c554b698271b> in <module>()
----> 1 pa.array([{'a': 1, 'b': 'string'}, {'b': 2}])

~/code/arrow/python/pyarrow/array.pxi in pyarrow.lib.array()
    179         if mask is not None:
    180             raise ValueError("Masks only supported with ndarray-like inputs")
--> 181         return _sequence_to_array(obj, size, type, pool)
    182 
    183 

~/code/arrow/python/pyarrow/array.pxi in pyarrow.lib._sequence_to_array()
     24         if size is None:
     25             with nogil:
---> 26                 check_status(ConvertPySequence(sequence, pool, &out))
     27         else:
     28             c_size = size

~/code/arrow/python/pyarrow/error.pxi in pyarrow.lib.check_status()
     89             raise ArrowNotImplementedError(message)
     90         elif status.IsTypeError():
---> 91             raise ArrowTypeError(message)
     92         elif status.IsCapacityError():
     93             raise ArrowCapacityError(message)

ArrowTypeError: ../src/arrow/python/builtin_convert.cc:794 code: AppendPySequence(seq, size, real_type, builder.get())
../src/arrow/python/iterators.h:60 code: func(value)
../src/arrow/python/builtin_convert.cc:619 code: value_converters_[i]->AppendSingle(valueobj ? valueobj : Py_None)
../src/arrow/python/builtin_convert.cc:414 code: internal::CIntFromPython(obj, &value)
../src/arrow/python/helpers.cc:259 code: CheckPyError()
an integer is required (got type str)

In [12]: pa.array([5, 'str', False])
---------------------------------------------------------------------------
ArrowTypeError                            Traceback (most recent call last)
<ipython-input-12-9e3343f08351> in <module>()
----> 1 pa.array([5, 'str', False])

~/code/arrow/python/pyarrow/array.pxi in pyarrow.lib.array()
    179         if mask is not None:
    180             raise ValueError("Masks only supported with ndarray-like inputs")
--> 181         return _sequence_to_array(obj, size, type, pool)
    182 
    183 

~/code/arrow/python/pyarrow/array.pxi in pyarrow.lib._sequence_to_array()
     24         if size is None:
     25             with nogil:
---> 26                 check_status(ConvertPySequence(sequence, pool, &out))
     27         else:
     28             c_size = size

~/code/arrow/python/pyarrow/error.pxi in pyarrow.lib.check_status()
     89             raise ArrowNotImplementedError(message)
     90         elif status.IsTypeError():
---> 91             raise ArrowTypeError(message)
     92         elif status.IsCapacityError():
     93             raise ArrowCapacityError(message)

ArrowTypeError: ../src/arrow/python/builtin_convert.cc:794 code: AppendPySequence(seq, size, real_type, builder.get())
../src/arrow/python/iterators.h:60 code: func(value)
../src/arrow/python/builtin_convert.cc:414 code: internal::CIntFromPython(obj, &value)
../src/arrow/python/helpers.cc:259 code: CheckPyError()
an integer is required (got type str)

Issue Links

• relates to: ARROW-3030 [Python] Create UnionArray from mixed-type pandas categorical