Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-13472

[R] Remove .engine = "duckdb" argument

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 6.0.0
    • Component/s: R
    • Labels:
      None

      Description

      ARROW-12688 added:

      • A new function to_duckdb() which registers an Arrow Dataset with DuckDB and returns a dbplyr object that can be used in dplyr pipelines
      • An .engine = "duckdb" option in the summarise() function which calls to_duckdb() inside summarise()

      At the moment, the latter is very convenient because summarise() is not yet natively supported for Arrow Datasets.

      However, this .engine = "duckdb" option is probably not such a great design for how users should interact with the arrow package in the longer term after native summarise() support is added. At that point, it will seem strange that this one particular dplyr verb has an .engine option while the others do not. Adding the option to all the other dplyr verbs also seems like a poor UX design.

      Consider whether we should ultimately have users choose whether to use the Arrow C++ engine or the DuckDB engine by passing an .engine argument to the collect() or compute() function, as Jonathan Keane suggested in these comments. collect() would return a tibble whereas compute() would return an Arrow Table.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jonkeane Jonathan Keane
                Reporter:
                icook Ian Cook
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: