[ARROW-14439] [Python][C++] Segfault with read_json when a field is missing

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.0.0
• Fix Version/s: 6.0.0
• Component/s: C++, Python
• External issue URL: https://github.com/apache/arrow/issues/18904
• Assignee: Unassigned
• Reporter: quentin lhoest
• Created: 22/Oct/21 14:05
• Updated: 11/Jan/23 08:39
• Resolved: 02/Nov/21 14:53

Description

When reading a JSON Lines file, a segfault can happen if there's a missing field at one point.
In particular when the missing field is supposed to be a list, and if the block size is small enough.

Here is an example to reproduce:

import io

import pyarrow.json as paj

batch = b'{"a": [], "b": 1}\n{"b": 1}'
block_size = 12

paj.read_json(
    io.BytesIO(batch), read_options=paj.ReadOptions(block_size=block_size)
)

Issue Links

• is superceded by: ARROW-13871 [C++] JSON reader can fail if a list array key is present in one chunk but not in a later chunk