[ARROW-12025] [Python] pyarrow read_csv works incorrectly with multilines if skiprows is present

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Not A Problem
• Affects Version/s: 3.0.0
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/18567
• Assignee: Unassigned
• Reporter: Alexander M
• Created: 19/Mar/21 14:53
• Updated: 11/Jan/23 08:23
• Resolved: 25/Jul/21 16:06

Description

Reproducer:
import os

from pyarrow.csv import read_csv, ReadOptions
import pyarrow
print("pyarrow._version:", pyarrow.version_)

test_filename = "test.csv"
test_data = """col1,col2,col3,col4
"This is a very long
string with several
newline characters",2,3,4
"""

try :
    with open(test_filename, "w") as f:
        f.write(test_data)

    ans_1 = read_csv(test_filename) # works fine
    print("ans_1: \n", ans_1)
    ans_2 = read_csv(test_filename, read_options=ReadOptions(skip_rows=2))
    print("ans_2: \n", ans_2)
finally:
    os.remove(test_filename)
 
Output:
pyarrow._version_: 3.0.0
ans_1:
pyarrow.Table
col1: string
col2: int64
col3: int64
col4: int64
Traceback (most recent call last):
File "pyarrow_bug.py", line 21, in <module>
ans_2 = read_csv(test_filename, read_options=ReadOptions(skip_rows=2))
File "pyarrow/_csv.pyx", line 714, in pyarrow._csv.read_csv
File "pyarrow/error.pxi", line 122, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: CSV parse error: Expected 1 columns, got 4
 
Note: python version: 3.8.8, platform: Ubuntu 20.04