[ARROW-2133] [Python] Segmentation fault on conversion of empty nested arrays to Pandas

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/18102
• Assignee: Antoine Pitrou
• Reporter: Uwe Korn
• Created: 11/Feb/18 21:30
• Updated: 11/Jan/23 07:19
• Resolved: 12/Feb/18 19:45

Description

After resolving https://issues.apache.org/jira/browse/ARROW-2129, we still cannot convert empty nested arrays to Pandas. It fails with the following backtrace:

(gdb) bt
#0 PyErr_Restore (type=type@entry=0x0, value=value@entry=0x0, traceback=traceback@entry=0x0) at Python/errors.c:42
#1 0x00007ffff7a1c48b in PyErr_Clear () at Python/errors.c:355
#2 0x00007ffff7959320 in PyDict_GetItem (op=0x7fffe831df78, key=key@entry=0x7fffec8873f0) at Objects/dictobject.c:1417
#3 0x00007ffff0278e89 in _buffer_clear_info (arr=0x7fffe37f52b0)
from /home/uwe/miniconda3/envs/pyarrow-dev/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so
#4 _array_dealloc_buffer_info (self=self@entry=0x7fffe37f52b0) at numpy/core/src/multiarray/buffer.c:776
#5 0x00007ffff022b0de in array_dealloc (self=0x7fffe37f52b0)
from /home/uwe/miniconda3/envs/pyarrow-dev/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so
#6 0x00007ffff172af72 in arrow::py::OwnedRef::reset (obj=0x0, this=0x7fffffff3760) at ../src/arrow/python/common.h:75
#7 arrow::py::OwnedRef::reset (this=0x7fffffff3760) at ../src/arrow/python/common.h:79
#8 0x00007ffff174220e in arrow::py::OwnedRef::~OwnedRef (this=0x7fffffff3760, __in_chrg=<optimized out>) at ../src/arrow/python/common.h:72
#9 arrow::py::ConvertListsLike<arrow::Int64Type> (options=..., col=std::shared_ptr (count 2, weak 0) 0x10b0ca0, out_values=<optimized out>,
out_values@entry=0xf793d0) at ../src/arrow/python/arrow_to_pandas.cc:522
#10 0x00007ffff173d833 in arrow::py::ArrowDeserializer::Visit (this=this@entry=0x7fffffff3aa0, type=...) at ../src/arrow/python/arrow_to_pandas.cc:1708
#11 0x00007ffff173dc0b in arrow::VisitTypeInline<arrow::py::ArrowDeserializer> (type=..., visitor=visitor@entry=0x7fffffff3aa0)
at ../src/arrow/visitor_inline.h:59
#12 0x00007ffff1728063 in arrow::py::ArrowDeserializer::Convert (out=0x7fffffff3ba8, this=0x7fffffff3aa0) at ../src/arrow/python/arrow_to_pandas.cc:1744
#13 arrow::py::ConvertColumnToPandas (options=..., options@entry=..., col=std::shared_ptr (count 2, weak 0) 0x10b0ca0, py_ref=py_ref@entry=0x7fffe37d4ea8,
out=out@entry=0x7fffffff3ba8) at ../src/arrow/python/arrow_to_pandas.cc:1769
#14 0x00007ffff17282e6 in arrow::py::ConvertArrayToPandas (options=..., arr=std::shared_ptr (count 2, weak 0) 0xfe9960, py_ref=0x7fffe37d4ea8,
out=0x7fffffff3ba8) at ../src/arrow/python/arrow_to_pandas.cc:1763
#15 0x00007ffff211b594 in __pyx_pw_7pyarrow_3lib_5Array_26to_pandas(_object*, _object*, _object*) ()
from /home/uwe/Development/arrow-repos-1/arrow/python/pyarrow/lib.cpython-36m-x86_64-linux-gnu.so
#16 0x00007ffff7967ded in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7ffff62921b0, args=args@entry=0x11d2bd8, nargs=<optimized out>,
kwargs=kwargs@entry=0x0) at Objects/methodobject.c:231

Issue Links

• links to: GitHub Pull Request #1591