Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Done
    • None
    • 9.0.0
    • Continuous Integration, R
    • None

    Description

      Find a way to host the R nightly binaries in some form of artifactory under ASF umbrella. Currently they are hosted on s3. See ARROW-16401

      Python wheels are hosted gemfury.io

      cc: Krisztian Szucs Kouhei Sutou Alessandro Molina Raúl Cumplido 

      Possible solutions, both of which could be extended for other Components that don't need an active server for their binary repos:

      • Host nightly builds on apache artifactory
        • https://apache.jfrog.io/artifactory
          • Is this even possible as nightlies are not signed official releases?
          • Size considerations (-> limit to 5 days as with conda?)
        • https://nightlies.apache.org/
          • Made for this purpose, extendable to other components like java that currently have no "nice" solution
          • Entirely under ASF control and on-brand{{}}
          • We could host M1 binaries compiled on Crossbow (by downloading them in arrow ci)
      • Host nightly builds on Github via Github Pages using either drat or manually creating the repo structure.
        • To prevent insane repo sizes this will require constant pruning/rewriting of git history 
        • Building and hosting on apache/arrow: 
          • The existing R CI could easily be extended to also commit the binaries to a gh_pages based repository, either on push to master or as a cron job. 
          • Entirely under ASF control and on-brand for users: install.packages("arrow", repos = "https://apache.github.io/arrow/r") vs. install.packages("arrow", repos = "https://arrow-r-nightly.s3.amazonaws.com")
          • Github Pages can be turned on via .asf.yaml - no ticket needed for Infra?
          • Not possible to create M1 binaries (for now?)
        • Hosting on a newly created utility repository apache/arrow-nightly:
          • Build binaries in Crossbow and upload via PAT with write access (not an option for apache/arrow due to security concerns)
          • We could host M1 binaries compiled on Crossbow
          • Entirely under ASF control and on-brand for users: install.packages("arrow", repos = "https://apache.github.io/arrow-nightly/") vs. install.packages("arrow", repos = "https://arrow-r-nightly.s3.amazonaws.com")
          • Does INFRA even allow such "utility" repos?
        • Building and hosting on ursacomputing/crossbow:
          • Neither under ASF control nor on-brand
          • We could host M1 binaries

      Attachments

        Issue Links

          Activity

            People

              assignuser Jacob Wujciak
              assignuser Jacob Wujciak
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Slack

                  Issue deployment