Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-15234

[Python] Possible crash with custom CSV invalid row handler

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      The crash happens on handler finalization and can be reproduced reliable here:

      $ taskset -c 1 python -m pytest -r s --tb=native pyarrow/tests/test_csv.py::TestThreadedCSVTableRead::test_invalid_row_handler
      =========================================================================== test session starts ===========================================================================
      platform linux -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
      rootdir: /home/antoine/arrow/dev/python, configfile: setup.cfg
      plugins: repeat-0.9.1, lazy-fixture-0.6.3, hypothesis-6.23.2
      collected 1 item                                                                                                                                                          
      
      pyarrow/tests/test_csv.py Fatal Python error: Segmentation fault
      
      Thread 0x00007fd0780af740 (most recent call first):
        File "/home/antoine/arrow/dev/python/pyarrow/tests/test_csv.py", line 670 in read_csv
        File "/home/antoine/arrow/dev/python/pyarrow/tests/test_csv.py", line 675 in read_bytes
        File "/home/antoine/arrow/dev/python/pyarrow/tests/test_csv.py", line 652 in test_invalid_row_handler
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/python.py", line 183 in pytest_pyfunc_call
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/python.py", line 1641 in runtest
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 162 in pytest_runtest_call
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 255 in <lambda>
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 311 in from_call
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 254 in call_runtest_hook
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 215 in call_and_report
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 126 in runtestprotocol
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/main.py", line 348 in pytest_runtestloop
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/main.py", line 323 in _main
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/main.py", line 269 in wrap_session
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/main.py", line 316 in pytest_cmdline_main
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/config/__init__.py", line 162 in main
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/_pytest/config/__init__.py", line 185 in console_main
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/site-packages/pytest/__main__.py", line 5 in <module>
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/runpy.py", line 87 in _run_code
        File "/home/antoine/miniconda3/envs/pyarrow/lib/python3.9/runpy.py", line 197 in _run_module_as_main
      Erreur de segmentation (core dumped)
      

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            apitrou Antoine Pitrou
            apitrou Antoine Pitrou
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h 20m
                1h 20m

                Slack

                  Issue deployment