[ARROW-2646] [C++/Python] Pandas roundtrip for date objects

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.11.0
• Component/s: C++, Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/19041
• Assignee: Krisztian Szucs
• Reporter: Florian Jetter
• Created: 30/May/18 09:03
• Updated: 11/Jan/23 07:22
• Resolved: 13/Sep/18 21:16

Description

Arrow currently casts date objects to nanosecond precision datetime objects. I'd like to have a way to preserve the type during a roundtrip

>>> import pandas as pd
>>> import pyarrow as pa
>>> import datetime
>>> pa.date32().to_pandas_dtype()
dtype('<M8[ns]')
>>> df = pd.DataFrame({'date': [datetime.date(2018, 1, 1)]})
>>> df.dtypes
date object
dtype: object
>>> df_rountrip = pa.Table.from_pandas(df).to_pandas()
>>> df_rountrip.dtypes
date    datetime64[ns]
dtype: object

I'd expect something like this to work:

>>> import pandas.testing as pdt
>>> df_rountrip = pa.Table.from_pandas(df).to_pandas(date_as_object=True)
>>> pdt.assert_frame_equal(df_rountrip, df)

Issue Links

• is duplicated by: ARROW-2679 pyarrow table from/to dataframe is type-lossy
• links to: GitHub Pull Request #2535