Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Done
-
None
-
None
Description
Find a way to host the R nightly binaries in some form of artifactory under ASF umbrella. Currently they are hosted on s3. See ARROW-16401
Python wheels are hosted gemfury.io
Possible solutions, both of which could be extended for other Components that don't need an active server for their binary repos:
- Host nightly builds on apache artifactory
- https://apache.jfrog.io/artifactory
- Is this even possible as nightlies are not signed official releases?
- Size considerations (-> limit to 5 days as with conda?)
- https://nightlies.apache.org/
Made for this purpose, extendable to other components like java that currently have no "nice" solutionEntirely under ASF control and on-brand{{}}We could host M1 binaries compiled on Crossbow (by downloading them in arrow ci)
- https://apache.jfrog.io/artifactory
- Host nightly builds on Github via Github Pages using either drat or manually creating the repo structure.
- To prevent insane repo sizes this will require constant pruning/rewriting of git history
- Building and hosting on apache/arrow:
- The existing R CI could easily be extended to also commit the binaries to a gh_pages based repository, either on push to master or as a cron job.
- Entirely under ASF control and on-brand for users: install.packages("arrow", repos = "https://apache.github.io/arrow/r") vs. install.packages("arrow", repos = "https://arrow-r-nightly.s3.amazonaws.com")
- Github Pages can be turned on via .asf.yaml - no ticket needed for Infra?
- Not possible to create M1 binaries (for now?)
- Hosting on a newly created utility repository apache/arrow-nightly:
- Build binaries in Crossbow and upload via PAT with write access (not an option for apache/arrow due to security concerns)
- We could host M1 binaries compiled on Crossbow
- Entirely under ASF control and on-brand for users: install.packages("arrow", repos = "https://apache.github.io/arrow-nightly/") vs. install.packages("arrow", repos = "https://arrow-r-nightly.s3.amazonaws.com")
- Does INFRA even allow such "utility" repos?
- Building and hosting on ursacomputing/crossbow:
- Neither under ASF control nor on-brand
- We could host M1 binaries
Attachments
Issue Links
- blocks
-
ARROW-16405 [R][CI] Use nightlies.apache.org as dev repo
- Resolved