[ARROW-10777] [Packaging][Python] PyPI pyarrow source dist (sdist) contains architecture dependent binaries

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 3.0.0
• Component/s: Packaging, Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/26718
• Assignee: Kouhei Sutou
• Reporter: Daniel Jewell
• Created: 30/Nov/20 21:42
• Updated: 11/Jan/23 08:15
• Resolved: 12/Jan/21 06:49

Description

Downloading the most recent pyarrow "sdist" source tarball from Pypi and upon extraction, the package contains multiple binary libraries compiled for x86-64. (libarrow, libparquet, etc.) 

 

The ultimate result is that this isn't a source package at all - it would be fine to include binaries in a python wheel but including arch/platform specific binaries in a sdist breaks pip and the install. (In my case, trying to install on aarch64.)

As a general observation, this will become a larger issue as, for example, the ARM-based Macs come to market. 

That said, one commonly implemented option is to make the python source package download and build any dependent libraries. 

 

At the very least, the source package should not contain binaries. I suppose it's not much different from a source Debian package containing compiled binary code.

 

Issue Links

• links to: GitHub Pull Request #9145