[ARROW-4774] [C++][Parquet] Call Table::Validate when writing a table

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.11.1, 0.12.1
• Fix Version/s: 0.13.0
• Component/s: C++
• Labels: pull-request-available
• Environment: Windows 10 16299.431, Python 3.6.4 64-bit, pyarrow 0.11.1 ; Windows Linux (WSL) Ubuntu 18.04.1, Python 3.6.5 64-bit, pyarrow 0.12.1
• External issue URL: https://github.com/apache/arrow/issues/21293
• Assignee: Francois Saint-Jacques
• Reporter: Stephen Gallagher
• Created: 05/Mar/19 11:19
• Updated: 11/Jan/23 07:36
• Resolved: 08/Mar/19 19:22

Description

When writing a table to a parquet file that contains both flat arrays of different leng

Reproducer:

import pyarrow as pa
import pyarrow.parquet as pq
import numpy as np

array1 = np.array([0, 1, 2], dtype=np.uint8)
array2 = np.array([[0,1,2], [3, 4, 5]], dtype=np.uint8).T

t1 = pa.uint8()
t2 = pa.list_(pa.uint8())

fields = [
    pa.field('a1', t1),
    pa.field('a2', t2)
]

myschema = pa.schema(fields)

mytable = pa.Table.from_arrays([
    pa.array(array1, type=t1),
    pa.array([array2[:,0], array2[:,1]], type=t2)],
    schema=myschema)

pq.write_table(mytable, 'example.parquet')

Windows 10 (Python 3.6.4 64-bit, pyarrow 0.11.1) crash code:

Process finished with exit code -1073741819 (0xC0000005)

WSL (Python 3.6.5 64-bit, pyarrow 0.12.1) Crash code:

Segmentation fault (core dumped)

Issue Links

• links to: GitHub Pull Request #3846