[PROTON-2095] Move away from SWIG to CFFI

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Done
• Affects Version/s: proton-c-0.29.0, proton-c-future
• Fix Version/s: proton-c-0.39.0
• Component/s: python-binding
• Assignee: Andrew Stitcher
• Reporter: Omer Katz
• Created: 30/Aug/19 10:30
• Updated: 07/Jun/23 12:18
• Resolved: 27/Jan/23 18:19

Description

SWIG is fine but we're not using it for anything other than exporting all of proton-c's API as is.

Unfortunately SWIG only generates CPython extension bindings. This may be a problem on PyPy where CPython extensions are either slow or simply won't compile.

Unlike SWIG, CFFI is portable both on CPython and PyPy.

It also satisfies the same requirements as SWIG currently does.

In addition, calls to CFFI simply release the GIL which will help parallelizing Python applications using threads.

By using CFFI we can also get rid of all of our setup.py code and simply use it to build the extension. We will also no longer have problems building wheels.

The newest version of CFFI supports pkg-config so we can use that to find proton-c easily.

I'm willing to help with the refactor but I'll need a mentor since I'm not familiar with the code base.

 

Issue Links

• causes: PROTON-2679 Parallel build of python bindings occasionally fails
• relates to: PROTON-2684 CFFI related TypeError: '%x format: an integer is required, not _cffi_backend._CDataBase'
• links to: github.com/swig/swig Support Python 3 stable API #1009