[ARROW-2593] [Python] TypeError: data type "mixed-integer" not understood

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.10.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/18978
• Assignee: Krisztian Szucs
• Reporter: Dima Ryazanov
• Created: 16/May/18 22:33
• Updated: 11/Jan/23 07:21
• Resolved: 29/May/18 12:49

Description

Pyarrow 0.9 raises an exception when converting some tables to pandas dataframes. Earlier versions work fine. Repro steps:

In [1]: import pandas as pd

In [2]: import pyarrow as pa

In [3]: df = pd.DataFrame({'foo': [], 123: []})

In [4]: table = pa.Table.from_pandas(df)

In [5]: table.to_pandas()
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/envs/cli3/lib/python3.6/site-packages/pyarrow/pandas_compat.py in _pandas_type_to_numpy_type(pandas_type)
    666     try:
--> 667         return _pandas_logical_type_map[pandas_type]
    668     except KeyError:

KeyError: 'mixed-integer'

(I ended up with a dataframe with mixed string/integer columns by using pd.read_excel(..., skiprows=[0]) - which skipped the header, and treated the first line of data as column names.)

Issue Links

• links to: GitHub Pull Request #2084