[ARROW-4311] [Python] Regression on pq.ParquetWriter incorrectly handling source string

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.12.0
• Fix Version/s: 0.13.0
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/15944
• Assignee: Antoine Pitrou
• Reporter: Francisco Sanchez
• Created: 21/Jan/19 14:59
• Updated: 11/Jan/23 07:33
• Resolved: 25/Mar/19 16:17

Description

In the latest changes to filesystem.py some new functions have been added to check the source string when calling pq.ParquetWriter. With the current implementation some assumptions are done about the format of the string which means that if the string is provided following some of these patterns it will be automatically split/formatted and changed to something else.

To give you a specific example, if I provide a string like directory/level1#level2.parquet it will be written to disk as directory/level1. The behaviour has changed on 0.12.0 from 0.11.1 and nothing is stated in the documentation.

Issue Links

• duplicates: ARROW-4559 [Python] pyarrow can't read/write filenames with special characters