[ARROW-10305] [R] Filter with regular expressions

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0
• Component/s: R
• External issue URL: https://github.com/apache/arrow/issues/26296
• Assignee: Unassigned
• Reporter: Pal
• Created: 14/Oct/20 06:09
• Updated: 11/Jan/23 08:12
• Resolved: 07/Apr/21 23:12

Description

Hi,

Some expressions, such as substr(), grepl(), str_detect() or others, are not supported while filtering a dataset (after open_datatset() ). Specifically, the code below :

library(dplyr)
library(arrow)
data = data.frame(a = c("a", "a2", "a3"))
write_parquet(data, "Test_filter/data.parquet")
ds <- open_dataset("Test_filter/")
data_flt <- ds %>% 
 filter(substr(a, 1, 1) == "a")

gives this error :

Error: Filter expression not supported for Arrow Datasets: substr(a, 1, 1) == "a"
 Call collect() first to pull data into R.

These expressions may be very helpful, not to say necessary, to filter and collect a very large dataset. Is there anything it can be done to implement this new feature ?

Thank you.

Issue Links

• depends upon: ARROW-9856 [R] Add bindings for string compute functions
• depends upon: ARROW-10195 [C++] Add string struct extract kernel using re2
• depends upon: ARROW-10557 [C++] Add scalar string slicing/substring extract kernel