Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2145

[Python] Decimal conversion not working for NaN values

    Details

      Description

      import pyarrow as pa
      import pandas as pd
      import decimal
      
      pa.Table.from_pandas(pd.DataFrame({'a': [decimal.Decimal('1.1'), decimal.Decimal('NaN')]}))
      

      throws following exception:

      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "pyarrow/table.pxi", line 875, in pyarrow.lib.Table.from_pandas (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:44927)
        File "/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 350, in dataframe_to_arrays
          convert_types)]
        File "/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 349, in <listcomp>
          for c, t in zip(columns_to_convert,
        File "/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 345, in convert_column
          return pa.array(col, from_pandas=True, type=ty)
        File "pyarrow/array.pxi", line 170, in pyarrow.lib.array (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:29224)
        File "pyarrow/array.pxi", line 70, in pyarrow.lib._ndarray_to_array (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:28465)
        File "pyarrow/error.pxi", line 98, in pyarrow.lib.check_status (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:9068)
      pyarrow.lib.ArrowException: Unknown error: an integer is required (got type str)
      

      Same problem with other special decimal values like infinity.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                cpcloud Phillip Cloud
                Reporter:
                antonymayi Antony Mayi
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: