Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2133

[Python] Segmentation fault on conversion of empty nested arrays to Pandas

    Details

      Description

      After resolving https://issues.apache.org/jira/browse/ARROW-2129, we still cannot convert empty nested arrays to Pandas. It fails with the following backtrace:

      (gdb) bt
      #0 PyErr_Restore (type=type@entry=0x0, value=value@entry=0x0, traceback=traceback@entry=0x0) at Python/errors.c:42
      #1 0x00007ffff7a1c48b in PyErr_Clear () at Python/errors.c:355
      #2 0x00007ffff7959320 in PyDict_GetItem (op=0x7fffe831df78, key=key@entry=0x7fffec8873f0) at Objects/dictobject.c:1417
      #3 0x00007ffff0278e89 in _buffer_clear_info (arr=0x7fffe37f52b0)
      from /home/uwe/miniconda3/envs/pyarrow-dev/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so
      #4 _array_dealloc_buffer_info (self=self@entry=0x7fffe37f52b0) at numpy/core/src/multiarray/buffer.c:776
      #5 0x00007ffff022b0de in array_dealloc (self=0x7fffe37f52b0)
      from /home/uwe/miniconda3/envs/pyarrow-dev/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so
      #6 0x00007ffff172af72 in arrow::py::OwnedRef::reset (obj=0x0, this=0x7fffffff3760) at ../src/arrow/python/common.h:75
      #7 arrow::py::OwnedRef::reset (this=0x7fffffff3760) at ../src/arrow/python/common.h:79
      #8 0x00007ffff174220e in arrow::py::OwnedRef::~OwnedRef (this=0x7fffffff3760, __in_chrg=<optimized out>) at ../src/arrow/python/common.h:72
      #9 arrow::py::ConvertListsLike<arrow::Int64Type> (options=..., col=std::shared_ptr (count 2, weak 0) 0x10b0ca0, out_values=<optimized out>,
      out_values@entry=0xf793d0) at ../src/arrow/python/arrow_to_pandas.cc:522
      #10 0x00007ffff173d833 in arrow::py::ArrowDeserializer::Visit (this=this@entry=0x7fffffff3aa0, type=...) at ../src/arrow/python/arrow_to_pandas.cc:1708
      #11 0x00007ffff173dc0b in arrow::VisitTypeInline<arrow::py::ArrowDeserializer> (type=..., visitor=visitor@entry=0x7fffffff3aa0)
      at ../src/arrow/visitor_inline.h:59
      #12 0x00007ffff1728063 in arrow::py::ArrowDeserializer::Convert (out=0x7fffffff3ba8, this=0x7fffffff3aa0) at ../src/arrow/python/arrow_to_pandas.cc:1744
      #13 arrow::py::ConvertColumnToPandas (options=..., options@entry=..., col=std::shared_ptr (count 2, weak 0) 0x10b0ca0, py_ref=py_ref@entry=0x7fffe37d4ea8,
      out=out@entry=0x7fffffff3ba8) at ../src/arrow/python/arrow_to_pandas.cc:1769
      #14 0x00007ffff17282e6 in arrow::py::ConvertArrayToPandas (options=..., arr=std::shared_ptr (count 2, weak 0) 0xfe9960, py_ref=0x7fffe37d4ea8,
      out=0x7fffffff3ba8) at ../src/arrow/python/arrow_to_pandas.cc:1763
      #15 0x00007ffff211b594 in __pyx_pw_7pyarrow_3lib_5Array_26to_pandas(_object*, _object*, _object*) ()
      from /home/uwe/Development/arrow-repos-1/arrow/python/pyarrow/lib.cpython-36m-x86_64-linux-gnu.so
      #16 0x00007ffff7967ded in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7ffff62921b0, args=args@entry=0x11d2bd8, nargs=<optimized out>,
      kwargs=kwargs@entry=0x0) at Objects/methodobject.c:231

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                pitrou Antoine Pitrou
                Reporter:
                xhochy Uwe L. Korn
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: