At the moment, we only test the conda based build in Travis but we also ship binary wheels after the release. The process of building them is currently part of the arrow-dist repository and uses the multibuild scripts that are used for many other Python packages that also have native code.
The code should be ported to run as a real CI job, i.e. in addition to just packaging the code, we will also need to run the unit tests. Furthermore, once the job is running and green, we also need to look at the runtimes as we already have a quite packed CI matrix and we expect that many steps of the wheel build are just to setup the environment. We should be able to cache them.
Maybe we want to do this as a nightly cron. For a first draft, it will be ok to add it to the full matrix.