[ARROW-3918] [Python] ParquetWriter.write_table doesn't support coerce_timestamps or allow_truncated_timestamps

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.11.1
• Fix Version/s: 0.12.0
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/20530
• Assignee: Krisztian Szucs
• Reporter: David Lee
• Created: 30/Nov/18 19:58
• Updated: 11/Jan/23 07:30
• Resolved: 10/Jan/19 11:56

Description

Error: Table Schema does not match schema used to create file.

The 0.11.1 release added these parameters to pyarrow.parquet.write_table(), but they are missing from pyarrow.parquet.ParquetWriter.write_table().. I'm seeing mismatches between the table schema and the file schema, but they are identical in the error message with modified: timestamp[ms] column types in both schemas. The only thing which looks odd is the Pandas metadata that has a modified column with a panda datatype of datetime and a numpy datatype of datetime64[ns]

 

Issue Links

• duplicates: ARROW-3728 [Python] Merging Parquet Files - Pandas Meta in Schema Mismatch