[ARROW-6923] [C++] Option for Filter kernel how to handle nulls in the selection vector

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: C++
• External issue URL: https://github.com/apache/arrow/issues/23246
• Assignee: Unassigned
• Reporter: Joris Van den Bossche
• Created: 17/Oct/19 18:49
• Updated: 11/Jan/23 07:50
• Resolved: 12/Jun/20 03:12

Description

How nulls are handled in the boolean mask (selection vector) in a filter kernel varies between languages / data analytics systems (e.g. base R propagates nulls, dplyr R skips (sees as False), SQL generally skips them as well I think, Julia raises an error).

Currently, in Arrow C++ we "propagate" nulls (null in the selection vector gives a null in the output):

In [7]: arr = pa.array([1, 2, 3]) 

In [8]: mask = pa.array([True, False, None]) 

In [9]: arr.filter(mask) 
Out[9]: 
<pyarrow.lib.Int64Array object at 0x7fefe44b3048>
[
  1,
  null
]

Given the different ways this could be done (propagate, skip, error), should we provide an option to control this behaviour?

Issue Links

• is a child of: ARROW-8894 [C++] C++ array kernels framework and execution buildout (umbrella issue)