[ARROW-4836] [Python] "Cannot tell() a compressed stream" when using RecordBatchStreamWriter

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.1
• Fix Version/s: 0.15.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/21350
• Assignee: Antoine Pitrou
• Reporter: Mike Pedersen
• Created: 12/Mar/19 11:36
• Updated: 11/Jan/23 07:36
• Resolved: 04/Sep/19 10:06

Description

It does not seem like RecordBatchStreamWriter works with compressed streams:

>>> import pyarrow as pa
>>> pa.__version__
'0.12.1'
>>> stream = pa.output_stream('/tmp/a.gz')
>>> batch = pa.RecordBatch.from_arrays([pa.array([1])], ['a'])
>>> writer = pa.RecordBatchStreamWriter(stream, batch.schema)
>>> writer.write(batch)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyarrow/ipc.pxi", line 181, in pyarrow.lib._RecordBatchWriter.write
  File "pyarrow/ipc.pxi", line 196, in pyarrow.lib._RecordBatchWriter.write_batch
  File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
pyarrow.lib.ArrowNotImplementedError: Cannot tell() a compressed stream

As I understand the documentation, this should be possible, right?

Issue Links

• links to: GitHub Pull Request #5262