[ARROW-6719] Parquet read_table error in Python3.7: pyarrow.lib.ArrowInvalid: Column data for field with type list<...> is inconsistent with schema list<...>

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Duplicate
• Affects Version/s: 0.15.0
• Labels: parquet
• Environment: Python 3.7
• External issue URL: https://github.com/apache/arrow/issues/23062
• Assignee: Unassigned
• Reporter: V Luong
• Created: 27/Sep/19 00:39
• Updated: 11/Jan/23 07:49
• Resolved: 15/Oct/19 17:39

Description

I have Parquet files with certain complex columns of type List<item: double>, List<item: string>, etc. and am using latest PyArrow (0.15.0) to process them.

In Python 3.7, the same pyarrow.parquet.read_table(...) function calls return errors of the following kind:

"pyarrow.lib.ArrowInvalid: Column data for field 0 with type list<item: double> is inconsistent with schema list<element: double>"

This issue might be related to https://issues.apache.org/jira/browse/ARROW-6068

Issue Links

• is duplicated by: ARROW-6844 [C++][Parquet][Python] List<scalar type> columns read broken with 0.15.0