[ARROW-2392] [Python] pyarrow RecordBatchStreamWriter allows writing batches with different schemas

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.13.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/15693
• Assignee: Antoine Pitrou
• Reporter: Ernesto Ocampo
• Created: 04/Apr/18 16:36
• Updated: 11/Jan/23 07:20
• Resolved: 27/Feb/19 18:29

Description

A RecordBatchStreamWriter initialised with a given schema will still allow writing RecordBatches that have different schemas. Example:

 

schema = pa.schema([pa.field('some_field', pa.int64())])
stream = pa.BufferOutputStream()
writer = pa.RecordBatchStreamWriter(stream, schema)

data = [pa.array([1.234])]
batch = pa.RecordBatch.from_arrays(data, ['some_field'])  
# batch does not conform to schema

assert batch.schema != schema

writer.write_batch(batch)  # no exception raised
writer.close()

Issue Links

• links to: GitHub Pull Request #3762