[ARROW-3030] [Python] Create UnionArray from mixed-type pandas categorical

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.10.0
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/19391
• Assignee: Unassigned
• Reporter: Christopher Brooks
• Created: 09/Aug/18 14:28
• Updated: 11/Jan/23 07:24
• Resolved: 04/Aug/21 13:22

Description

While troublehsooting ARROW-2966 I updated my pandas dataframe with more type information. Specifically, I changed some mixed type columns to categorical instead of object. I assumed that the Table.from_pandas() would inspect the pandas type information and respect that when converting it over to a table. It doesn't seem to.

For instance, I expected this code to work, but it throws the same ArrowTypeError as ARROW-2966.

 

import pandas as pd
import pyarrow as pa
import numpy as np
df=pd.DataFrame.from_dict({"col":[0,1,2,3,""]},dtype="category")
tb = pa.Table.from_pandas(df, columns=["col"])

 

Issue Links

• is related to: ARROW-2774 [Python] Implement pa.array() with type=union type