[ARROW-17130] Enable multiple character delimiters in read_csv

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 8.0.1
• Component/s: Format
• External issue URL: https://github.com/apache/arrow/issues/32432
• Assignee: Unassigned
• Reporter: Jack Howard
• Created: 19/Jul/22 16:06
• Updated: 11/Jan/23 11:48

Description

Read_CSV ParseOptions allows only a single character delimiter. Single character delimiters are highly susceptible to the candidate value existing within the data to be loaded, negating the ability to serve as a delimiter.

If a double character delimiter is used, the current limit of a single character returns "only single character unicode strings can be converted to Py_UCS4, got length 2"