[ARROW-2940] [Python] Import error with pytorch 0.3

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/15761
• Assignee: Unassigned
• Reporter: Philipp Moritz
• Created: 30/Jul/18 22:42
• Updated: 11/Jan/23 07:24
• Resolved: 30/Jul/18 23:50

Description

The fix in ARROW-2920 doesn't work in versions strictly before pytorch 0.4:

>>> import pyarrow

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/home/ubuntu/arrow/python/pyarrow/__init__.py", line 57, in <module>

    compat.import_pytorch_extension()

  File "/home/ubuntu/arrow/python/pyarrow/compat.py", line 249, in import_pytorch_extension

    ctypes.CDLL(os.path.join(path, "lib/libcaffe2.so"))

  File "/home/ubuntu/anaconda3/envs/breaking-env2/lib/python3.5/ctypes/__init__.py", line 351, in __init__

    self._handle = _dlopen(self._name, mode)

OSError: /home/ubuntu/anaconda3/envs/breaking-env2/lib/python3.5/site-packages/torch/lib/libcaffe2.so: cannot open shared object file: No such file or directory

Issue Links

• links to: GitHub Pull Request #2342