Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-15234

[Python] Possible crash with custom CSV invalid row handler

    XMLWordPrintableJSON

Details

    Description

      The crash happens on handler finalization and can be reproduced reliable here:

      $ taskset -c 1 python -m pytest -r s --tb=native pyarrow/tests/test_csv.py::TestThreadedCSVTableRead::test_invalid_row_handler
      =========================================================================== test session starts ===========================================================================
      platform linux -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
      rootdir: /home/antoine/arrow/dev/python, configfile: setup.cfg
      plugins: repeat-0.9.1, lazy-fixture-0.6.3, hypothesis-6.23.2
      collected 1 item                                                                                                                                                          
      
      pyarrow/tests/test_csv.py Fatal Python error: Segmentation fault
      
      Thread 0x00007fd0780af740 (most recent call first):
        File "/home/antoine/arrow/dev/python/pyarrow/tests/test_csv.py", line 670 in read_csv
        File "/home/antoine/arrow/dev/python/pyarrow/tests/test_csv.py", line 675 in read_bytes
        File "/home/antoine/arrow/dev/python/pyarrow/tests/test_csv.py", line 652 in test_invalid_row_handler
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/python.py", line 183 in pytest_pyfunc_call
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/python.py", line 1641 in runtest
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 162 in pytest_runtest_call
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 255 in <lambda>
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 311 in from_call
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 254 in call_runtest_hook
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 215 in call_and_report
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 126 in runtestprotocol
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/main.py", line 348 in pytest_runtestloop
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/main.py", line 323 in _main
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/main.py", line 269 in wrap_session
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/main.py", line 316 in pytest_cmdline_main
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/config/__init__.py", line 162 in main
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/config/__init__.py", line 185 in console_main
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pytest/__main__.py", line 5 in <module>
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/runpy.py", line 87 in _run_code
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/runpy.py", line 197 in _run_module_as_main
      Erreur de segmentation (core dumped)
      

      Attachments

        Issue Links

          Activity

            People

              apitrou Antoine Pitrou
              apitrou Antoine Pitrou
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m