Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-4836

[Python] "Cannot tell() a compressed stream" when using RecordBatchStreamWriter

    XMLWordPrintableJSON

    Details

      Description

      It does not seem like RecordBatchStreamWriter works with compressed streams:

      >>> import pyarrow as pa
      >>> pa.__version__
      '0.12.1'
      >>> stream = pa.output_stream('/tmp/a.gz')
      >>> batch = pa.RecordBatch.from_arrays([pa.array([1])], ['a'])
      >>> writer = pa.RecordBatchStreamWriter(stream, batch.schema)
      >>> writer.write(batch)
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "pyarrow/ipc.pxi", line 181, in pyarrow.lib._RecordBatchWriter.write
        File "pyarrow/ipc.pxi", line 196, in pyarrow.lib._RecordBatchWriter.write_batch
        File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
      pyarrow.lib.ArrowNotImplementedError: Cannot tell() a compressed stream
      

      As I understand the documentation, this should be possible, right?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                apitrou Antoine Pitrou
                Reporter:
                mpedersen Mike Pedersen
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m