[ARROW-2263] [Python] test_cython.py fails if pyarrow is not in import path (e.g. with inplace builds)

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/18221
• Assignee: Wes McKinney
• Reporter: Wes McKinney
• Created: 05/Mar/18 19:46
• Updated: 11/Jan/23 07:19
• Resolved: 09/Mar/18 04:04

Description

see

$ py.test pyarrow/tests/test_cython.py 
===================================== test session starts =====================================
platform linux -- Python 3.6.4, pytest-3.4.1, py-1.5.2, pluggy-0.6.0
rootdir: /home/wesm/code/arrow/python, inifile: setup.cfg
collected 1 item                                                                              

pyarrow/tests/test_cython.py F                                                          [100%]

========================================== FAILURES ===========================================
_______________________________________ test_cython_api _______________________________________

tmpdir = local('/tmp/pytest-of-wesm/pytest-3/test_cython_api0')

    @pytest.mark.skipif(
        'ARROW_HOME' not in os.environ,
        reason='ARROW_HOME environment variable not defined')
    def test_cython_api(tmpdir):
        """
        Basic test for the Cython API.
        """
        pytest.importorskip('Cython')
    
        ld_path_default = os.path.join(os.environ['ARROW_HOME'], 'lib')
    
        test_ld_path = os.environ.get('PYARROW_TEST_LD_PATH', ld_path_default)
    
        with tmpdir.as_cwd():
            # Set up temporary workspace
            pyx_file = 'pyarrow_cython_example.pyx'
            shutil.copyfile(os.path.join(here, pyx_file),
                            os.path.join(str(tmpdir), pyx_file))
            # Create setup.py file
            if os.name == 'posix':
                compiler_opts = ['-std=c++11']
            else:
                compiler_opts = []
            setup_code = setup_template.format(pyx_file=pyx_file,
                                               compiler_opts=compiler_opts,
                                               test_ld_path=test_ld_path)
            with open('setup.py', 'w') as f:
                f.write(setup_code)
    
            # Compile extension module
            subprocess.check_call([sys.executable, 'setup.py',
>                                  'build_ext', '--inplace'])

pyarrow/tests/test_cython.py:90: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

popenargs = (['/home/wesm/miniconda/envs/arrow-dev/bin/python', 'setup.py', 'build_ext', '--inplace'],)
kwargs = {}, retcode = 1
cmd = ['/home/wesm/miniconda/envs/arrow-dev/bin/python', 'setup.py', 'build_ext', '--inplace']

    def check_call(*popenargs, **kwargs):
        """Run command with arguments.  Wait for command to complete.  If
        the exit code was zero then return, otherwise raise
        CalledProcessError.  The CalledProcessError object will have the
        return code in the returncode attribute.
    
        The arguments are the same as for the call function.  Example:
    
        check_call(["ls", "-l"])
        """
        retcode = call(*popenargs, **kwargs)
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
>           raise CalledProcessError(retcode, cmd)
E           subprocess.CalledProcessError: Command '['/home/wesm/miniconda/envs/arrow-dev/bin/python', 'setup.py', 'build_ext', '--inplace']' returned non-zero exit status 1.

../../../miniconda/envs/arrow-dev/lib/python3.6/subprocess.py:291: CalledProcessError
------------------------------------ Captured stderr call -------------------------------------
Traceback (most recent call last):
  File "setup.py", line 7, in <module>
    import pyarrow as pa
ModuleNotFoundError: No module named 'pyarrow'
================================== 1 failed in 0.23 seconds ===================================

I encountered this bit of brittleness in a fresh install where I had not run setup.py develop nor setup.py install on my local pyarrow dev area

Issue Links

• links to: GitHub Pull Request #1730