Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-5161

[Python] Cannot convert struct type from Pandas object column

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • 0.13.0
    • 0.15.0
    • Python
    • None

    Description

      As reported on https://github.com/apache/arrow/issues/4045. Interestingly, the datatype is inferred correctly.

      >>> df = pd.DataFrame({'col': [{'ints': 5, 'strs': 'foo'}, {'ints': 6, 'strs': 'bar'}]})                                                                             
      >>> df                                                                                                                                                               
                                col
      0  {'ints': 5, 'strs': 'foo'}
      1  {'ints': 6, 'strs': 'bar'}
      >>> pa.Table.from_pandas(df)                                                                                                                                         
      Traceback (most recent call last):
        File "<ipython-input-16-6eac5d0eec08>", line 1, in <module>
          pa.Table.from_pandas(df)
        File "pyarrow/table.pxi", line 1139, in pyarrow.lib.Table.from_pandas
          names, arrays, metadata = dataframe_to_arrays(
        File "/home/antoine/arrow/dev/python/pyarrow/pandas_compat.py", line 480, in dataframe_to_arrays
          types)
        File "/home/antoine/arrow/dev/python/pyarrow/pandas_compat.py", line 209, in construct_metadata
          field_name=sanitized_name)
        File "/home/antoine/arrow/dev/python/pyarrow/pandas_compat.py", line 151, in get_column_metadata
          logical_type = get_logical_type(arrow_type)
        File "/home/antoine/arrow/dev/python/pyarrow/pandas_compat.py", line 79, in get_logical_type
          raise NotImplementedError(str(arrow_type))
      NotImplementedError: struct<ints: int64, strs: string>
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            apitrou Antoine Pitrou
            Votes:
            2 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment