Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-11328

[R] Collecting zero columns from a dataset returns entire dataset

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0, 3.0.0
    • 4.0.0
    • R

    Description

      Collecting a dataset with zero selected columns returns all columns of the dataset in a data frame without column names.

      library(dplyr)
      #> 
      #> Attaching package: 'dplyr'
      #> The following objects are masked from 'package:stats':
      #> 
      #>     filter, lag
      #> The following objects are masked from 'package:base':
      #> 
      #>     intersect, setdiff, setequal, union
      library(arrow)
      #> 
      #> Attaching package: 'arrow'
      #> The following object is masked from 'package:utils':
      #> 
      #>     timestamp
      
      tmp <- tempfile()
      write_dataset(mtcars, tmp, format = "parquet")
      open_dataset(tmp) %>% select() %>% collect()
      #>                                             
      #> 1  21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4
      #> 2  21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4
      #> 3  22.8 4 108.0  93 3.85 2.320 18.61 1 1 4 1
      #> 4  21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1
      #> 5  18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2
      #> 6  18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1
      #> 7  14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4
      #> 8  24.4 4 146.7  62 3.69 3.190 20.00 1 0 4 2
      #> 9  22.8 4 140.8  95 3.92 3.150 22.90 1 0 4 2
      #> 10 19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4
      #> 11 17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4
      #> 12 16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3
      #> 13 17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3
      #> 14 15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3
      #> 15 10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4
      #> 16 10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
      #> 17 14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
      #> 18 32.4 4  78.7  66 4.08 2.200 19.47 1 1 4 1
      #> 19 30.4 4  75.7  52 4.93 1.615 18.52 1 1 4 2
      #> 20 33.9 4  71.1  65 4.22 1.835 19.90 1 1 4 1
      #> 21 21.5 4 120.1  97 3.70 2.465 20.01 1 0 3 1
      #> 22 15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
      #> 23 15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
      #> 24 13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
      #> 25 19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
      #> 26 27.3 4  79.0  66 4.08 1.935 18.90 1 1 4 1
      #> 27 26.0 4 120.3  91 4.43 2.140 16.70 0 1 5 2
      #> 28 30.4 4  95.1 113 3.77 1.513 16.90 1 1 5 2
      #> 29 15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
      #> 30 19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
      #> 31 15.0 8 301.0 335 3.54 3.570 14.60 0 1 5 8
      #> 32 21.4 4 121.0 109 4.11 2.780 18.60 1 1 4 2
      

      Empty selections in dplyr return data frames with zero columns and based on test cases covering dplyr verbs on RecordBatches already handle empty selections in the same way.

      Created on 2021-01-20 by the reprex package (v0.3.0)

      Session info

      devtools::session_info()
      #> ─ Session info ───────────────────────────────────────────────────────────────
      #>  setting  value                       
      #>  version  R version 4.0.3 (2020-10-10)
      #>  os       Ubuntu 20.04.1 LTS          
      #>  system   x86_64, linux-gnu           
      #>  ui       X11                         
      #>  language (EN)                        
      #>  collate  en_US.UTF-8                 
      #>  ctype    en_US.UTF-8                 
      #>  tz       Etc/UTC                     
      #>  date     2021-01-20                  
      #> 
      #> - Packages -------------------------------------------------------------------
      #>  package     * version        date       lib source        
      #>  arrow       * 2.0.0.20210119 2021-01-20 [1] local         
      #>  assertthat    0.2.1          2019-03-21 [1] RSPM (R 4.0.0)
      #>  bit           4.0.4          2020-08-04 [1] RSPM (R 4.0.2)
      #>  bit64         4.0.5          2020-08-30 [1] RSPM (R 4.0.2)
      #>  callr         3.5.1          2020-10-13 [1] RSPM (R 4.0.2)
      #>  cli           2.2.0          2020-11-20 [1] CRAN (R 4.0.3)
      #>  crayon        1.3.4          2017-09-16 [1] RSPM (R 4.0.0)
      #>  DBI           1.1.1          2021-01-15 [1] CRAN (R 4.0.3)
      #>  desc          1.2.0          2018-05-01 [1] RSPM (R 4.0.0)
      #>  devtools      2.3.2          2020-09-18 [1] RSPM (R 4.0.2)
      #>  digest        0.6.27         2020-10-24 [1] RSPM (R 4.0.3)
      #>  dplyr       * 1.0.3          2021-01-15 [1] CRAN (R 4.0.3)
      #>  ellipsis      0.3.1          2020-05-15 [1] RSPM (R 4.0.0)
      #>  evaluate      0.14           2019-05-28 [1] RSPM (R 4.0.0)
      #>  fansi         0.4.2          2021-01-15 [1] CRAN (R 4.0.3)
      #>  fs            1.5.0          2020-07-31 [1] RSPM (R 4.0.2)
      #>  generics      0.1.0          2020-10-31 [1] CRAN (R 4.0.3)
      #>  glue          1.4.2          2020-08-27 [1] RSPM (R 4.0.2)
      #>  highr         0.8            2019-03-20 [1] RSPM (R 4.0.0)
      #>  htmltools     0.5.1          2021-01-12 [1] RSPM (R 4.0.3)
      #>  knitr         1.30           2020-09-22 [1] CRAN (R 4.0.2)
      #>  lifecycle     0.2.0          2020-03-06 [1] RSPM (R 4.0.0)
      #>  magrittr      2.0.1          2020-11-17 [1] RSPM (R 4.0.3)
      #>  memoise       1.1.0          2017-04-21 [1] RSPM (R 4.0.0)
      #>  pillar        1.4.7          2020-11-20 [1] CRAN (R 4.0.3)
      #>  pkgbuild      1.2.0          2020-12-15 [1] RSPM (R 4.0.3)
      #>  pkgconfig     2.0.3          2019-09-22 [1] RSPM (R 4.0.0)
      #>  pkgload       1.1.0          2020-05-29 [1] RSPM (R 4.0.0)
      #>  prettyunits   1.1.1          2020-01-24 [1] RSPM (R 4.0.0)
      #>  processx      3.4.5          2020-11-30 [1] RSPM (R 4.0.3)
      #>  ps            1.5.0          2020-12-05 [1] CRAN (R 4.0.3)
      #>  purrr         0.3.4          2020-04-17 [1] RSPM (R 4.0.0)
      #>  R6            2.5.0          2020-10-28 [1] RSPM (R 4.0.3)
      #>  remotes       2.2.0          2020-07-21 [1] RSPM (R 4.0.2)
      #>  rlang         0.4.10         2020-12-30 [1] CRAN (R 4.0.3)
      #>  rmarkdown     2.6            2020-12-14 [1] RSPM (R 4.0.3)
      #>  rprojroot     2.0.2          2020-11-15 [1] RSPM (R 4.0.3)
      #>  sessioninfo   1.1.1          2018-11-05 [1] RSPM (R 4.0.0)
      #>  stringi       1.5.3          2020-09-09 [1] RSPM (R 4.0.2)
      #>  stringr       1.4.0          2019-02-10 [1] RSPM (R 4.0.0)
      #>  testthat      3.0.1          2020-12-17 [1] RSPM (R 4.0.3)
      #>  tibble        3.0.5          2021-01-15 [1] CRAN (R 4.0.3)
      #>  tidyselect    1.1.0          2020-05-11 [1] RSPM (R 4.0.0)
      #>  usethis       2.0.0          2020-12-10 [1] RSPM (R 4.0.3)
      #>  vctrs         0.3.6          2020-12-17 [1] RSPM (R 4.0.3)
      #>  withr         2.4.0          2021-01-16 [1] CRAN (R 4.0.3)
      #>  xfun          0.20           2021-01-06 [1] RSPM (R 4.0.3)
      #>  yaml          2.2.1          2020-02-01 [1] RSPM (R 4.0.0)
      #> 
      #> [1] /usr/local/lib/R/site-library
      #> [2] /usr/lib/R/site-library
      #> [3] /usr/lib/R/library
      

      Attachments

        Issue Links

          Activity

            People

              pachamaltese Mauricio 'Pachá' Vargas Sepúlveda
              svraka András Svraka
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h
                  1h