Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-5019

[C#] ArrowStreamWriter doesn't work on a non-seekable stream

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: C#

      Description

      When writing to a non-seekable .NET Stream (like a network/socket stream), ArrowStreamWriter will throw an exception:

       

      Exception thrown: 'System.NotSupportedException' in System.Net.Sockets.dll
      This stream does not support seek operations.
      

      The reason this throws is because we are using `BastStream.Position` in the writer to calculate the length of bytes that we've written to the stream. We don't need to use the Position in order to calculate the lengths. We should be able to write an Arrow RecordBatch to a NetworkStream directly. Today, we need to write to a MemoryStream, and then copy the MemoryStream to the NetworkStream.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                eerhardt Eric Erhardt
                Reporter:
                eerhardt Eric Erhardt
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 48h
                  48h
                  Remaining:
                  Time Spent - 26.5h Remaining Estimate - 21.5h
                  21.5h
                  Logged:
                  Time Spent - 26.5h Remaining Estimate - 21.5h
                  26.5h