Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-5161

[Python] Cannot convert struct type from Pandas object column

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 0.13.0
    • Fix Version/s: 0.15.0
    • Component/s: Python
    • Labels:
      None

      Description

      As reported on https://github.com/apache/arrow/issues/4045. Interestingly, the datatype is inferred correctly.

      >>> df = pd.DataFrame({'col': [{'ints': 5, 'strs': 'foo'}, {'ints': 6, 'strs': 'bar'}]})                                                                             
      >>> df                                                                                                                                                               
                                col
      0  {'ints': 5, 'strs': 'foo'}
      1  {'ints': 6, 'strs': 'bar'}
      >>> pa.Table.from_pandas(df)                                                                                                                                         
      Traceback (most recent call last):
        File "<ipython-input-16-6eac5d0eec08>", line 1, in <module>
          pa.Table.from_pandas(df)
        File "pyarrow/table.pxi", line 1139, in pyarrow.lib.Table.from_pandas
          names, arrays, metadata = dataframe_to_arrays(
        File "/home/antoine/arrow/dev/python/pyarrow/pandas_compat.py", line 480, in dataframe_to_arrays
          types)
        File "/home/antoine/arrow/dev/python/pyarrow/pandas_compat.py", line 209, in construct_metadata
          field_name=sanitized_name)
        File "/home/antoine/arrow/dev/python/pyarrow/pandas_compat.py", line 151, in get_column_metadata
          logical_type = get_logical_type(arrow_type)
        File "/home/antoine/arrow/dev/python/pyarrow/pandas_compat.py", line 79, in get_logical_type
          raise NotImplementedError(str(arrow_type))
      NotImplementedError: struct<ints: int64, strs: string>
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              apitrou Antoine Pitrou
            • Votes:
              2 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: