[ARROW-9664] [Python] Array/ChunkedArray.to_pandas do not support types_mapper keyword

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 8.0.0
• Component/s: Python
• Labels: pull-request-available
• Environment: pyarrow: 1.0.0 ; pandas: 1.0.5 ; python: sys.version_info(major=3, minor=8, micro=2, releaselevel='final', serial=0)
• External issue URL: https://github.com/apache/arrow/issues/25723
• Assignee: Alenka Frim
• Reporter: Adrien Hoarau
• Created: 06/Aug/20 14:33
• Updated: 11/Jan/23 08:08
• Resolved: 15/Feb/22 10:14

Description

Arrow structures (Array, ChunkedArray, Table) have a types_mapper argument in their to_pandas method. It works for Table, but doesn't seem to get called for Array or ChunkedArray:

import pandas as pd
import pyarrow

data = pd.Series([0, None, 2], dtype=pd.Int32Dtype(), name='foo')

def convert_types(arrow_type):
     raise ValueError("Function got called")


pyarrow.Table.from_pandas(data.to_frame()).to_pandas(types_mapper=convert_types)

Traceback (most recent call last):
  File "/home/adrien/.pyenv/versions/complete/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3331, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-6-d1e3e1f45f69>", line 1, in <module>
    pyarrow.Table.from_pandas(data.to_frame()).to_pandas(types_mapper=convert_types)
  File "pyarrow/array.pxi", line 715, in pyarrow.lib._PandasConvertible.to_pandas
  File "pyarrow/table.pxi", line 1565, in pyarrow.lib.Table._to_pandas
  File "/home/adrien/.pyenv/versions/complete/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 771, in table_to_blockmanager
    ext_columns_dtypes = _get_extension_dtypes(
  File "/home/adrien/.pyenv/versions/complete/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 840, in _get_extension_dtypes
    pandas_dtype = types_mapper(typ)
  File "<ipython-input-5-5a9760e8753f>", line 2, in convert_types
    raise ValueError("Function got called")
ValueError: Function got called

pyarrow.Int32Array.from_pandas(data).to_pandas(types_mapper=convert_types)

0    0.0
1    NaN
2    2.0
dtype: float64

Issue Links

• is duplicated by: ARROW-9787 [Python] pa.array.to_pandas(types_mapper=...) doesn't change the dtype
• links to: GitHub Pull Request #12178