[ARROW-2658] [Python] Serialize and Deserialize Table objects

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/19052
• Assignee: Unassigned
• Reporter: Kunal Gosar
• Created: 01/Jun/18 01:37
• Updated: 11/Jan/23 07:22
• Resolved: 29/Aug/22 14:14

Description

Add support for serializing and deserializing pyarrow Tables. This would allow using Table objects in plasma and DataFrames can be converted to a Table object as intermediary for serialization. Currently I see the following when trying this operation:

In [36]: pa.serialize(t)

---------------------------------------------------------------------------

SerializationCallbackError                Traceback (most recent call last)

<ipython-input-36-48cee5dfe30a> in <module>()

----> 1 pa.serialize(t)



~/dev/arrow/python/pyarrow/serialization.pxi in pyarrow.lib.serialize()

    336

    337     with nogil:

--> 338         check_status(SerializeObject(context, wrapped_value, &serialized.data))

    339     return serialized

    340



~/dev/arrow/python/pyarrow/serialization.pxi in pyarrow.lib.SerializationContext._serialize_callback()

    134

    135         if not found:

--> 136             raise SerializationCallbackError(

    137                 "pyarrow does not know how to "

    138                 "serialize objects of type {}.".format(type(obj)), obj



SerializationCallbackError: pyarrow does not know how to serialize objects of type <class 'pyarrow.lib.Table'>.