Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-4135

[Python] Can't reload a pandas dataframe containing a list of datetime.time

    XMLWordPrintableJSON

Details

    Description

      Ported from https://github.com/apache/arrow/issues/3223

      This simple script writes a panda dataframe with a list of datetime.time. However, constructing back the pandas dataframe fails.

      I initially realised that when doing a pd.read_parquet('example.parquet'), which fails with the same error.

      This is using
      pyarrow 0.11.1
      pandas 0.23.4

      import pandas as pd
      import pyarrow as pa
      import pyarrow.parquet as pq
      
      times = pd.to_datetime(['09:00', '09:30', '10:00', '10:30', '11:00', '11:30', '12:00']).time
      df = pd.DataFrame({'Time': [times]})
      
      table = pa.Table.from_pandas(df)
      pq.write_table(table, 'example.parquet')
      
      # works
      table2 = pq.read_table('example.parquet')
      
      # fails: ArrowNotImplementedError: Not implemented type for list in DataFrameBlock: time64[us]
      df2 = pa.Table.to_pandas(table2)
      

      Attachments

        Issue Links

          Activity

            People

              kszucs Krisztian Szucs
              kszucs Krisztian Szucs
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m