[ARROW-10173] [Rust][DataFusion] Improve performance of equality to a constant predicate support

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Rust, Rust - DataFusion
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/26181
• Assignee: Yordan Pavlov
• Reporter: Andrew Lamb
• Created: 05/Oct/20 11:13
• Updated: 11/Jan/23 08:11
• Resolved: 18/Nov/20 04:48

Description

I noticed this behavior while working on support for DictionaryArrays and wanted to capture it in a ticket in case someone has time to work on it.

In order to implement an equality predicate to a constant such as d1 = 'three', DataFusion effectively creates an array with the same value 'three' repeated over and over again and uses the equality compute kernel. This is ... suboptimal.

Here is what the predicate looks like:

        predicate: BinaryExpr {
            left: CastExpr {
                expr: Column {
                    name: "d1",
                },
                cast_type: Utf8,
            },
            op: Eq,
            right: Literal {
                value: Utf8("three"),
            },
        },

Issue Links

• relates to: ARROW-8907 [Rust] implement scalar comparison operations
• relates to: ARROW-8908 [Rust][DataFusion] improve performance of building literal arrays
• links to: GitHub Pull Request #8660