[ARROW-10025] [Rust] Filter kernel relies on bits outside of valid len of boolean array

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.1
• Component/s: Rust, Rust - DataFusion
• External issue URL: https://github.com/apache/arrow/issues/26048
• Assignee: Unassigned
• Reporter: Jörn Horstmann
• Created: 16/Sep/20 15:57
• Updated: 11/Jan/23 08:10
• Resolved: 24/Sep/20 18:16

Description

The filter kernel accesses the filter array in chunks of u64 and tries to access elements if the corresponding bit is set. This assumes that bits outside of the valid len of that array have to be unset. That assumption is currently not correct for output of the NOT and the IS NULL kernels, which also operate on and negate a whole chunk of their input arrays.

Testcase (depending on is null kernel from ARROW-10016)

#[test]
fn csv_query_with_is_null_predicate() -> Result<()> {
    let mut ctx = ExecutionContext::new();
    register_aggregate_csv(&mut ctx)?;
    let sql = "SELECT COUNT(1) FROM aggregate_test_100 WHERE c1 IS NULL";
    let actual = execute(&mut ctx, sql).join("\n");
    let expected = "0".to_string();
    assert_eq!(expected, actual);
    Ok(())
}

 

Issue Links

• is fixed by: ARROW-10016 [Rust] [DataFusion] Implement IsNull and IsNotNull