[ARROW-9096] [Python] Pandas roundtrip with object-dtype column labels with integer values: data type "integer" not understood

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.17.1
• Fix Version/s: 2.0.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/25210
• Assignee: Andrew Wieteska
• Reporter: Richard Wu
• Created: 10/Jun/20 23:34
• Updated: 11/Jan/23 08:04
• Resolved: 03/Aug/20 18:35

Description

The following will fail the roundtrip since the column indexes' pandas_type is converted from int64 to integer when an additional column is introduced and subsequently moved to the index:

 

df = pd.DataFrame(np.ones((3,1), index=[[1,2,3]])
df['foo'] = np.arange(3)
df = df.set_index('foo', append=True)
table = pyarrow.Table.from_pandas(df)
table.to_pandas()  # Errors

Issue Links

• links to: GitHub Pull Request #7822