Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-3842

[R] RecordBatchStreamWriter api

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.12.0
    • R

    Description

      To support the "Writing and Reading Streams" section of the vignette, perhaps we should rely more on the RecordBatchStreamWriter class and less the `write_record_batch` function.

      We should be able to write code resembling the python api :

      batch <- ... 
      sink <- buffer_output_stream()
      writer <- record_batch_stream_writer(sink, batch$schema())
      writer$write_batch()
      writer$close()
      sink$getvalue()
      

      Most of the code is there, but we need to add

      • RecordBatchStreamWriter$write_batch() : write a record batch to the stream. We already have RecordBatchStreamWriter$WriteRecordBatch
      • RecordBatchStreamWriter$close() : not sure why it is lower case close() in python but upper case in C++. We already have RecordBatchWriter$Close()
      • BufferOutputStream$getvalue() : we already have BufferOutputStream$Finish()

      Currently the constructor for a BufferOutputStream is buffer_output_stream(), perhaps we can align with python and make it BufferOutputStream, that would not clash with the `arrow::BufferOutputStream` class because of the namespacing.

      Attachments

        Issue Links

          Activity

            People

              romainfrancois Romain Francois
              romainfrancois Romain Francois
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h