[ARROW-6038] [Python] pyarrow.Table.from_batches produces corrupted table if any of the batches were empty

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.13.0, 0.14.0, 0.14.1
• Fix Version/s: 0.15.0
• Component/s: C++, Python
• Labels: pull-request-available windows
• External issue URL: https://github.com/apache/arrow/issues/22441
• Assignee: Antoine Pitrou
• Reporter: Piotr Bajger
• Created: 25/Jul/19 16:09
• Updated: 11/Jan/23 07:44
• Resolved: 16/Aug/19 02:29

Description

When creating a Table from a list/iterator of batches which contains an "empty" RecordBatch a Table is produced but attempts to run any pyarrow built-in functions (such as unique()) occasionally result in a Segfault.

The MWE is attached: segfault_ex.py

1. The segfaults happen randomly, around 30% of the time.
2. Commenting out line 10 in the MWE results in no segfaults.
3. The segfault is triggered using the unique() function, but I doubt the behaviour is specific to that function, from what I gather the problem lies in Table creation.

I'm on Windows 10, using Python 3.6 and pyarrow 0.14.0 installed through pip (problem also occurs with 0.13.0 from conda-forge).

Issue Links

• links to: GitHub Pull Request #4983