Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2273

Cannot deserialize pandas SparseDataFrame

    XMLWordPrintableJSON

    Details

      Description

      >>> import pyarrow
      >>> import pandas
      >>> a = pandas.SparseDataFrame(

      {'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]}

      )
      >>> pyarrow.deserialize(pyarrow.serialize(a).to_buffer())
      Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "serialization.pxi", line 441, in pyarrow.lib.deserialize
      File "serialization.pxi", line 404, in pyarrow.lib.deserialize_from
      File "serialization.pxi", line 257, in pyarrow.lib.SerializedPyObject.deserialize
      File "serialization.pxi", line 174, in pyarrow.lib.SerializationContext._deserialize_callback
      File ".../.virtualenv/arrow/lib/python3.6/site-packages/pyarrow/serialization.py", line 77, in _deserialize_pandas_dataframe
      return pdcompat.serialized_dict_to_dataframe(data)
      File ".../.virtualenv/arrow/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 450, in serialized_dict_to_dataframe
      for block in data['blocks']]
      File ".../.virtualenv/arrow/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 450, in <listcomp>
      for block in data['blocks']]
      File ".../.virtualenv/arrow/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 478, in _reconstruct_block
      block = _int.make_block(block_arr, placement=placement)
      File ".../.virtualenv/arrow/lib/python3.6/site-packages/pandas/core/internals.py", line 2957, in make_block
      return klass(values, ndim=ndim, fastpath=fastpath, placement=placement)
      File ".../.virtualenv/arrow/lib/python3.6/site-packages/pandas/core/internals.py", line 120, in _init_
      len(self.mgr_locs)))
      ValueError: Wrong number of items passed 3, placement implies 1

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Licht-T Licht Takeuchi
                Reporter:
                mitar Mitar
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m