[IGNITE-13967] Refactor and improve performance of python thin client marshaller

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: thin client
• Labels: python
• Ignite Flags: Docs Required, Release Notes Required
• Assignee: Ivan Daschinsky
• Reporter: Ivan Daschinsky
• Created: 11/Jan/21 11:41
• Updated: 10/Feb/21 10:35
• Resolved: 08/Feb/21 14:13

Description

Currently implemented serialization has questionable design and suffers from some problems
1. It is tightly coupled with Client object
2. It doesn't use protocol feature that total length of message is in the header,
thus it constantly load from Client some data instead of iteration over byte array.
3. It uses some tricky hacks and sometimes new connection is created when deserializing object.
4. It constantly allocates bytes (immutable data structure).

I suggest to rewrite serialization and deserialization:
1. Pass to corresponding methods specific SerDe context + BytesIO
2. Context can be sync and async and contains specific flags and methods for loading/uploading binary object schemas
3. Refactor Client in order to retrieve full packet from socket at once then pass full packet futher.

These steps can significantly improve performance, reduce amount of allocations and give
foundation for implementing asyncio version of client.

Issue Links

• blocks: IGNITE-13911 asyncio version of python ignite thin client
• links to: GitHub Pull Request #10
• links to: GitHub Pull Request #14
• links to: TC build