[ARROW-15362] Setting OMP_NUM_THREADS to 1 causes dataset to hang

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 6.0.1
• Fix Version/s: 7.0.0
• Component/s: C++
• External issue URL: https://github.com/apache/arrow/issues/30847
• Assignee: Weston Pace
• Reporter: Devin Petersohn
• Created: 18/Jan/22 18:38
• Updated: 11/Jan/23 11:36
• Resolved: 18/Jan/22 19:45

Description

import os
import pyarrow.dataset as ds

os.environ["OMP_NUM_THREADS"] = "1"  # offending line

dfile = 'any.csv'
data = ds.dataset(dfile, format='csv')
rb_iter = iter(data.to_batches())
next(rb_iter)  # hangs