[ARROW-2145] [Python] Decimal conversion not working for NaN values

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.9.0
• Component/s: C++, Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/18112
• Assignee: Phillip Cloud
• Reporter: Antony Mayi
• Created: 13/Feb/18 07:19
• Updated: 11/Jan/23 07:19
• Resolved: 01/Mar/18 22:27

Description

import pyarrow as pa
import pandas as pd
import decimal

pa.Table.from_pandas(pd.DataFrame({'a': [decimal.Decimal('1.1'), decimal.Decimal('NaN')]}))

throws following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyarrow/table.pxi", line 875, in pyarrow.lib.Table.from_pandas (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:44927)
  File "/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 350, in dataframe_to_arrays
    convert_types)]
  File "/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 349, in <listcomp>
    for c, t in zip(columns_to_convert,
  File "/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 345, in convert_column
    return pa.array(col, from_pandas=True, type=ty)
  File "pyarrow/array.pxi", line 170, in pyarrow.lib.array (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:29224)
  File "pyarrow/array.pxi", line 70, in pyarrow.lib._ndarray_to_array (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:28465)
  File "pyarrow/error.pxi", line 98, in pyarrow.lib.check_status (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:9068)
pyarrow.lib.ArrowException: Unknown error: an integer is required (got type str)

Same problem with other special decimal values like infinity.

Issue Links

• is related to: ARROW-2153 [C++/Python] Decimal conversion not working for exponential notation
• links to: GitHub Pull Request #1610
• links to: GitHub Pull Request #1651