[ARROW-16641] [R] How to filter array columns?

• Type: Wish
• Status: Closed
• Priority: Minor
• Resolution: Information Provided
• Component/s: R
• External issue URL: https://github.com/apache/arrow/issues/31991
• Assignee: Will Jones
• Reporter: Will Jones
• Created: 24/May/22 13:55
• Updated: 11/Jan/23 11:45
• Resolved: 30/Jun/22 16:18

Description

In the parquet data we have, there is a column with the array data type (list<array_element <string>>), which flags records that have different issues. For each record, multiple values could be stored in the column. For example, `[A, B, C]`.

I'm trying to perform a data filtering step and exclude some flagged records.

Filtering is trivial for the regular columns that contain just a single value. E.g.,

flags_to_exclude <- c("A", "B")
datt %>% filter(! col %in% flags_to_exclude)

Given the array column, is it possible to exclude records with at least one of the flags from `flags_to_exclude` using the arrow R package?

I really appreciate any advice you can provide!

Issue Links

• relates to: ARROW-16702 [C++] Add compute functions for list array containment