Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2658

[Python] Serialize and Deserialize Table objects

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Python
    • Labels:
      None

      Description

      Add support for serializing and deserializing pyarrow Tables. This would allow using Table objects in plasma and DataFrames can be converted to a Table object as intermediary for serialization. Currently I see the following when trying this operation:

      In [36]: pa.serialize(t)
      
      ---------------------------------------------------------------------------
      
      SerializationCallbackError                Traceback (most recent call last)
      
      <ipython-input-36-48cee5dfe30a> in <module>()
      
      ----> 1 pa.serialize(t)
      
      
      
      ~/dev/arrow/python/pyarrow/serialization.pxi in pyarrow.lib.serialize()
      
          336
      
          337     with nogil:
      
      --> 338         check_status(SerializeObject(context, wrapped_value, &serialized.data))
      
          339     return serialized
      
          340
      
      
      
      ~/dev/arrow/python/pyarrow/serialization.pxi in pyarrow.lib.SerializationContext._serialize_callback()
      
          134
      
          135         if not found:
      
      --> 136             raise SerializationCallbackError(
      
          137                 "pyarrow does not know how to "
      
          138                 "serialize objects of type {}.".format(type(obj)), obj
      
      
      
      SerializationCallbackError: pyarrow does not know how to serialize objects of type <class 'pyarrow.lib.Table'>.
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              kunalgosar Kunal Gosar
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: