[ARROW-3062] [Python] Extend fast libtensorflow_framework.so compatibility workaround to Python 2.7

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0
• Fix Version/s: 0.11.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/19420
• Assignee: Philipp Moritz
• Reporter: Philipp Moritz
• Created: 15/Aug/18 21:35
• Updated: 11/Jan/23 07:24
• Resolved: 16/Aug/18 14:05

Description

The workaround ARROW-2657 should be optimized a little bit and use the loading of libtensorflow_framework.so (instead of doing a full "import tensorflow") also for Python 2.7.

We are running into this, since doing "import tensorflow" spawns a number of threads, so without this optimization, using many python processes with pyarrow will hit OS limits for threads.

Issue Links

• links to: GitHub Pull Request #2435