[ARROW-14177] [C++] Optimize dictionary support in kernels/Support nulls in DictionaryUnifier

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: C++
• Labels: kernel
• External issue URL: https://github.com/apache/arrow/issues/29764
• Assignee: Unassigned
• Reporter: David Li
• Created: 29/Sep/21 19:48
• Updated: 11/Jan/23 08:38

Description

ARROW-13358, ARROW-14167, ARROW-13573, etc. add support for dictionary types in various generic 'selection'-style kernels. However, the support is fairly unoptimized, repeatedly hashing and inserting the same values. Instead, we could implement more optimized support in some cases.

For example, for if_else, we could first unify both dictionaries (in such a way that the first dictionary is unchanged), then copy-with-transpose the indices of the second argument, then copy the relevant indices of the first argument (which doesn't require transposition). This would require DictionaryUnifier to support nulls in dictionaries, however.

Issue Links

• is related to: ARROW-14649 [R] Include unused factor levels in coalesce() and if_else() output
• relates to: ARROW-13358 [C++] Extend type support for if_else kernel
• relates to: ARROW-13573 [C++] Support dictionaries directly in case_when kernel
• relates to: ARROW-14167 [C++] Support dictionaries directly in coalesce kernel