Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-5863

[Python] Segmentation Fault via pytest-runner

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.14.0
    • 0.14.1, 0.15.0
    • Python

    Description

      When running pytest on projects using pyarrow==0.14.0 on Linux, I am getting segmentation faults, but interestingly only when run via pytest-runner (which provides the setup.py pytest command)

      This works (i.e. pytest directly):

      $ pytest
      
      Test session starts (platform: linux, Python 3.7.3, pytest 5.0.0, pytest-sugar 0.9.2)
      benchmark: 3.2.2 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
      rootdir: /home/josh/scratch/pyarrow-issue
      plugins: sugar-0.9.2, Flask-Dance-2.2.0, env-0.6.2, mock-1.10.4, xdist-1.29.0, requests-mock-1.6.0, forked-1.0.2, dash-1.0.0, cov-2.7.1, html-1.21.1, benchmark-3.2.2, metadata-1.8.0
      collecting ...
      tests/test_pyarrow.py ✓ 100% ██████████
      
      Results (0.09s):
      1 passed

      However, this does not work, ending in a segmentation fault, even though the tests pass:

      $ python setup.py pytest
      
      running pytest
      running egg_info
      writing pyarrow_issue.egg-info/PKG-INFO
      writing dependency_links to pyarrow_issue.egg-info/dependency_links.txt
      writing requirements to pyarrow_issue.egg-info/requires.txt
      writing top-level names to pyarrow_issue.egg-info/top_level.txt
      reading manifest file 'pyarrow_issue.egg-info/SOURCES.txt'
      writing manifest file 'pyarrow_issue.egg-info/SOURCES.txt'
      running build_ext
      
      Test session starts (platform: linux, Python 3.7.3, pytest 5.0.0, pytest-sugar 0.9.2)
      benchmark: 3.2.2 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
      rootdir: /home/josh/scratch/pyarrow-issue
      plugins: sugar-0.9.2, Flask-Dance-2.2.0, env-0.6.2, mock-1.10.4, xdist-1.29.0, requests-mock-1.6.0, forked-1.0.2, dash-1.0.0, cov-2.7.1, html-1.21.1, benchmark-3.2.2, metadata-1.8.0
      collecting ...
      tests/test_pyarrow.py ✓ 100% ██████████
      
      Results (0.07s):
      1 passed
      zsh: segmentation fault (core dumped) python setup.py pytest

      backtrace from gdb

      Thread 1 "python" received signal SIGSEGV, Segmentation fault.
      0x00007ffff7c10b58 in ?? () from /usr/lib/libpython3.7m.so.1.0
      
      (gdb) bt
      #0 0x00007ffff7c10b58 in ?? () from /usr/lib/libpython3.7m.so.1.0
      #1 0x00007ffff7ae46cc in ?? () from /usr/lib/libpython3.7m.so.1.0
      #2 0x00007ffff023a6b3 in arrow::py::PyExtensionType::~PyExtensionType() ()
      from /home/josh/.virtualenvs/default/lib/python3.7/site-packages/pyarrow/./libarrow_python.so.14
      #3 0x00007fffed5e6467 in std::unordered_map<std::string, std::shared_ptr<arrow::ExtensionType>, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, std::shared_ptr<arrow::ExtensionType> > > >::~unordered_map() ()
      from /home/josh/.virtualenvs/default/lib/python3.7/site-packages/pyarrow/./libarrow.so.14
      #4 0x00007ffff7de5e70 in __run_exit_handlers () from /usr/lib/libc.so.6
      #5 0x00007ffff7de5fae in exit () from /usr/lib/libc.so.6
      #6 0x00007ffff7dcfeea in __libc_start_main () from /usr/lib/libc.so.6
      #7 0x000055555555505e in _start ()
      

      I have observed this behaviour on my machine running natively, and also via docker. Also, 0.13.0 does not exhibit this behaviour

       

      Attachments

        1. pyarrow-issue.tar.bz2
          0.4 kB
          Josh Bode
        2. pytest.log
          3 kB
          Josh Bode
        3. pytest-runner.log
          5 kB
          Josh Bode

        Issue Links

          Activity

            People

              wesm Wes McKinney
              joshbode Josh Bode
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m