Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-4836

"Cannot tell() a compressed stream" when using RecordBatchStreamWriter

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.12.1
    • Fix Version/s: None
    • Component/s: Python
    • Labels:
      None

      Description

      It does not seem like RecordBatchStreamWriter works with compressed streams:

      >>> import pyarrow as pa
      >>> pa.__version__
      '0.12.1'
      >>> stream = pa.output_stream('/tmp/a.gz')
      >>> batch = pa.RecordBatch.from_arrays([pa.array([1])], ['a'])
      >>> writer = pa.RecordBatchStreamWriter(stream, batch.schema)
      >>> writer.write(batch)
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "pyarrow/ipc.pxi", line 181, in pyarrow.lib._RecordBatchWriter.write
        File "pyarrow/ipc.pxi", line 196, in pyarrow.lib._RecordBatchWriter.write_batch
        File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
      pyarrow.lib.ArrowNotImplementedError: Cannot tell() a compressed stream
      

      As I understand the documentation, this should be possible, right?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              mpedersen Mike Pedersen
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: