[ARROW-8638] Arrow Cython API Usage Gives an error when calling CTable API Endpoints

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Information Provided
• Affects Version/s: 0.16.0
• Fix Version/s: 0.16.0
• Component/s: C++, Python
• Environment: Ubuntu 20.04 with Python 3.8.2 ; RHEL7 with Python 3.6.8
• External issue URL: https://github.com/apache/arrow/issues/17158
• Assignee: Unassigned
• Reporter: Vibhatha Lakmal Abeykoon
• Created: 30/Apr/20 03:22
• Updated: 11/Jan/23 08:01
• Resolved: 30/Apr/20 14:16

Description

I am working on using both Arrow C++ API and Cython API to support an application that I am developing. But here, I will add the issue I experienced when I am trying to follow the example, 

https://arrow.apache.org/docs/python/extending.html

I am testing on Ubuntu 20.04 LTS

Python version 3.8.2

These are the steps I followed.

1. Create Virtualenv

python3 -m venv ENVARROW

 

2. Activate ENV

source ENVARROW/bin/activate

 

3. pip3 install pyarrow==0.16.0 cython numpy

 

 4. Code block and Tools,

 

example.pyx

 

 

from pyarrow.lib cimport *
def get_array_length(obj):
 # Just an example function accessing both the pyarrow Cython API
 # and the Arrow C++ API
 cdef shared_ptr[CArray] arr = pyarrow_unwrap_array(obj)
 if arr.get() == NULL:
 raise TypeError("not an array")
 return arr.get().length()
def get_table_info(obj):
 cdef shared_ptr[CTable] table = pyarrow_unwrap_table(obj)
 if table.get() == NULL:
 raise TypeError("not an table")
 
 return table.get().num_columns() 

 

 

setup.py

 

 

from distutils.core import setup
from Cython.Build import cythonize
import os
import numpy as np
import pyarrow as pa

ext_modules = cythonize("example.pyx")
for ext in ext_modules:
 # The Numpy C headers are currently required
 ext.include_dirs.append(np.get_include())
 ext.include_dirs.append(pa.get_include())
 ext.libraries.extend(pa.get_libraries())
 ext.library_dirs.extend(pa.get_library_dirs())
if os.name == 'posix':
 ext.extra_compile_args.append('-std=c++11')
# Try uncommenting the following line on Linux
 # if you get weird linker errors or runtime crashes
 #ext.define_macros.append(("_GLIBCXX_USE_CXX11_ABI", "0"))

setup(ext_modules=ext_modules)

 

 

arrow_array.py

 

import example
import pyarrow as pa
import numpy as np
arr = pa.array([1,2,3,4,5])
len = example.get_array_length(arr)
print("Array length {} ".format(len)) 

 

arrow_table.py

 

import example
import pyarrow as pa
import numpy as np
from pyarrow import csv
fn = 'data.csv'
table = csv.read_csv(fn)
print(table)
cols = example.get_table_info(table)
print(cols)
 

data.csv

1,2,3,4,5
6,7,8,9,10
11,12,13,14,15

 

Makefile

 

install: 
        python3 setup.py build_ext --inplace
clean: 
        rm -R *.so build *.cpp 

 

**When I try to run either of the python example scripts arrow_table.py or arrow_array.py, 

I get the following error. 

 

File "arrow_array.py", line 1, in <module>
 import example
ImportError: libarrow.so.16: cannot open shared object file: No such file or directory

 

 

Note: I also checked this on RHEL7 with Python 3.6.8, I got a similar response.