Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-1187

Serialize a DataFrame with None column

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.4.1
    • Fix Version/s: 0.5.0
    • Component/s: Python
    • Labels:
      None

      Description

      import pandas as pd
      import feather
      pd.__version__
      feather.version.version
      df = pd.DataFrame({"A": (1, 2, 3), "B": (None, None, None)})
      feather.write_dataframe(df, "/tmp/foo.feather")
      

      results in:

      ---------------------------------------------------------------------------
      ArrowNotImplementedError                  Traceback (most recent call last)
      <ipython-input-72-799e35eece97> in <module>()
            6 df
            7 df.info()
      ----> 8 feather.write_dataframe(df, "/tmp/foo.feather")
      
      /home/feanor/.virtualenvs/xbrl/lib/python3.6/site-packages/pyarrow/feather.py in write_feather(df, dest)
          110     writer = FeatherWriter(dest)
          111     try:
      --> 112         writer.write(df)
          113     except:
          114         # Try to make sure the resource is closed
      
      /home/feanor/.virtualenvs/xbrl/lib/python3.6/site-packages/pyarrow/feather.py in write(self, df)
           99                 name = str(name)
          100 
      --> 101             self.writer.write_array(name, col)
          102 
          103         self.writer.close()
      
      pyarrow/io.pxi in pyarrow.lib.FeatherWriter.write_array (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:48584)()
      
      pyarrow/error.pxi in pyarrow.lib.check_status (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:7124)()
      
      ArrowNotImplementedError: NotImplemented: null
      

      The versions are:

      pandas: '0.19.2'
      arrow: '0.4.0'
      

        Attachments

          Activity

            People

            • Assignee:
              xhochy Uwe L. Korn
              Reporter:
              pmav99 Panagiotis Mavrogiorgos
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: