[ARROW-1187] Serialize a DataFrame with None column

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4.1
• Fix Version/s: 0.5.0
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/16910
• Assignee: Uwe Korn
• Reporter: Panagiotis Mavrogiorgos
• Created: 06/Jul/17 12:23
• Updated: 11/Jan/23 07:13
• Resolved: 13/Jul/17 12:45

Description

import pandas as pd
import feather
pd.__version__
feather.version.version
df = pd.DataFrame({"A": (1, 2, 3), "B": (None, None, None)})
feather.write_dataframe(df, "/tmp/foo.feather")

results in:

---------------------------------------------------------------------------
ArrowNotImplementedError                  Traceback (most recent call last)
<ipython-input-72-799e35eece97> in <module>()
      6 df
      7 df.info()
----> 8 feather.write_dataframe(df, "/tmp/foo.feather")

/home/feanor/.virtualenvs/xbrl/lib/python3.6/site-packages/pyarrow/feather.py in write_feather(df, dest)
    110     writer = FeatherWriter(dest)
    111     try:
--> 112         writer.write(df)
    113     except:
    114         # Try to make sure the resource is closed

/home/feanor/.virtualenvs/xbrl/lib/python3.6/site-packages/pyarrow/feather.py in write(self, df)
     99                 name = str(name)
    100 
--> 101             self.writer.write_array(name, col)
    102 
    103         self.writer.close()

pyarrow/io.pxi in pyarrow.lib.FeatherWriter.write_array (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:48584)()

pyarrow/error.pxi in pyarrow.lib.check_status (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:7124)()

ArrowNotImplementedError: NotImplemented: null

The versions are:

pandas: '0.19.2'
arrow: '0.4.0'