Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-1187

Serialize a DataFrame with None column

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.4.1
    • 0.5.0
    • Python
    • None

    Description

      import pandas as pd
      import feather
      pd.__version__
      feather.version.version
      df = pd.DataFrame({"A": (1, 2, 3), "B": (None, None, None)})
      feather.write_dataframe(df, "/tmp/foo.feather")
      

      results in:

      ---------------------------------------------------------------------------
      ArrowNotImplementedError                  Traceback (most recent call last)
      <ipython-input-72-799e35eece97> in <module>()
            6 df
            7 df.info()
      ----> 8 feather.write_dataframe(df, "/tmp/foo.feather")
      
      /home/feanor/.virtualenvs/xbrl/lib/python3.6/site-packages/pyarrow/feather.py in write_feather(df, dest)
          110     writer = FeatherWriter(dest)
          111     try:
      --> 112         writer.write(df)
          113     except:
          114         # Try to make sure the resource is closed
      
      /home/feanor/.virtualenvs/xbrl/lib/python3.6/site-packages/pyarrow/feather.py in write(self, df)
           99                 name = str(name)
          100 
      --> 101             self.writer.write_array(name, col)
          102 
          103         self.writer.close()
      
      pyarrow/io.pxi in pyarrow.lib.FeatherWriter.write_array (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:48584)()
      
      pyarrow/error.pxi in pyarrow.lib.check_status (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:7124)()
      
      ArrowNotImplementedError: NotImplemented: null
      

      The versions are:

      pandas: '0.19.2'
      arrow: '0.4.0'
      

      Attachments

        Activity

          People

            uwe Uwe Korn
            pmav99 Panagiotis Mavrogiorgos
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: