Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-13472

[R] Remove .engine = "duckdb" argument

    XMLWordPrintableJSON

Details

    Description

      ARROW-12688 added:

      • A new function to_duckdb() which registers an Arrow Dataset with DuckDB and returns a dbplyr object that can be used in dplyr pipelines
      • An .engine = "duckdb" option in the summarise() function which calls to_duckdb() inside summarise()

      At the moment, the latter is very convenient because summarise() is not yet natively supported for Arrow Datasets.

      However, this .engine = "duckdb" option is probably not such a great design for how users should interact with the arrow package in the longer term after native summarise() support is added. At that point, it will seem strange that this one particular dplyr verb has an .engine option while the others do not. Adding the option to all the other dplyr verbs also seems like a poor UX design.

      Consider whether we should ultimately have users choose whether to use the Arrow C++ engine or the DuckDB engine by passing an .engine argument to the collect() or compute() function, as jonkeane suggested in these comments. collect() would return a tibble whereas compute() would return an Arrow Table.

      Attachments

        Issue Links

          Activity

            People

              jonkeane Jonathan Keane
              icook Ian Cook
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m