[ARROW-8223] [Python] Schema.from_pandas breaks with pandas nullable integer dtype

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 0.15.0, 0.15.1, 0.16.0
• Fix Version/s: 0.17.0
• Component/s: Python
• Labels: easyfix
• Environment: pyarrow 0.16
• External issue URL: https://github.com/apache/arrow/issues/24420
• Assignee: Uwe Korn
• Reporter: Ged Steponavicius
• Created: 25/Mar/20 22:41
• Updated: 11/Jan/23 07:58
• Resolved: 26/Mar/20 10:53

Description

 

import pandas as pd
import pyarrow as pa
df = pd.DataFrame([{'int_col':1},
 {'int_col':2}])
df['int_col'] = df['int_col'].astype(pd.Int64Dtype())

schema = pa.Schema.from_pandas(df)

produces ArrowTypeError: Did not pass numpy.dtype object

 

However, this works fine 

schema = pa.Table.from_pandas(df).schema

Issue Links

• is duplicated by: ARROW-8159 [Python] pyarrow.Schema.from_pandas doesn't support ExtensionDtype