[ARROW-8719] Stream data easily- function to convert record batch to streamable format

• Type: Wish
• Status: Closed
• Priority: Minor
• Resolution: Information Provided
• Component/s: C
• Labels: buffer data-transfer socket
• Environment: Linux
• External issue URL: https://github.com/apache/arrow/issues/24872
• Assignee: Unassigned
• Reporter: Steven
• Created: 06/May/20 17:09
• Updated: 11/Jan/23 08:02
• Resolved: 06/May/20 20:59

Description

I am currently working on a project that  need to send the data in different processes. Instead of storing it in a file, I am using the IPC stream method, storing it into a buffer. When I try to send the data(bytes) through a socket, I loose the data in the client side. ( Which gives me the idea that when i leave the function(create buffer in the server side folder), I loose acces to that data. Can there be an example that explains this.

Also, can you implement a function that convert the record batch into the format that can be sent through the socket directly.

 

The code are in the attachment, one for the server side code and the other for the client side code.

Thank you