Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-8487

[FlightRPC][C++] Make it possible to target a specific payload size

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      gRPC by default limits message sizes on the wire. While Flight in turn disables these by default, they're still useful to be able to control memory consumption. A well-behaved client/server may wish to split up writes to respect these limits. However, right now, there's no way to measure the memory usage of what you're about to write without serializing it.

      With ARROW-5377, we can in theory avoid this by having the writer take control of serialization, producing the IpcPayload, then measuring the size and writing the payload if the size is as desired. However, Flight doesn't provide such a low-level mechanism yet - we'd need to open that up as well.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            lidavidm David Li Assign to me
            lidavidm David Li
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h
                2h

                Slack

                  Issue deployment