Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-6107

[Go] ipc.Writer Option to skip appending data buffers

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Go
    • Labels:
      None

      Description

      For cases where we have a known shared memory region, it would be great if the ipc.Writer (and by extension ipc.Reader?) had the ability to write out everything but the actual buffers holding the data. That way we can still utilize the ipc mechanisms to communicate without having to serialize all the underlying data across the wire.

       

      This seems like it should be possible since the `RecordBatch` flatbuffers only contain the metadata and the underlying data buffers are appended later. We just need to skip appending the underlying data buffers.

       

      Sebastien Binet thoughts?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              nickpoorman Nick Poorman
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: