[ARROW-5295] [Python] accept pyarrow values / scalars in constructor functions ?

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Python
• Labels: python-conversion
• External issue URL: https://github.com/apache/arrow/issues/21761
• Assignee: Unassigned
• Reporter: Joris Van den Bossche
• Created: 09/May/19 16:23
• Updated: 11/Jan/23 07:39

Description

Currently, functions like pyarrow.array don't accept pyarrow Arrays, or also not scalars of it:

In [42]: arr = pa.array([1, 2, 3])

In [43]: pa.array(arr)
...
ArrowInvalid: Could not convert 1 with type pyarrow.lib.Int64Value: did not recognize Python value type when inferring an Arrow data type

In [44]: pa.array(list(arr))
...
ArrowInvalid: Could not convert 1 with type pyarrow.lib.Int64Value: did not recognize Python value type when inferring an Arrow data type

Do we want to allow those / recognize those here? (the first case could even have a fastpath, as we don't need to do it element by element).

Also scalars are not supported:

In [46]: type(arr.sum())
Out[46]: pyarrow.lib.Int64Scalar

In [47]: pa.array([arr.sum()])
...
ArrowInvalid: Could not convert 6 with type pyarrow.lib.Int64Scalar: did not recognize Python value type when inferring an Arrow data type

And also in other functions we don't accept arrow scalars / values:

In [48]: string = pa.array(['a'])[0]

In [49]: type(string)
Out[49]: pyarrow.lib.StringValue

In [50]: pa.field(string, pa.int64())
...
TypeError: expected bytes, pyarrow.lib.StringValue found

 

 

Issue Links

• duplicates: ARROW-10935 [Python] pa.array() doesn't support pa.lib.TimestampScalar objects
• is duplicated by: ARROW-18432 [Python] Array constructor doesn't support arrow scalars.
• relates to: ARROW-16254 [Python] Scalar constructors should be callable
• relates to: ARROW-16645 [Python] Allow pa.array/pa.chunked_array to infer pa.NA when in a non pyarrow container