[SPARK-47693] Improve UTF8_BINARY_LCASE collation comparison performance

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available
• Epic Link: Collation support
• Assignee: Nikola Mandic
• Reporter: Nikola Mandic
• Created: 02/Apr/24 10:17
• Updated: 29/Apr/24 16:12
• Resolved: 10/Apr/24 15:24

Description

Current collation benchmarks indicate that UTF8_BINARY_LCASE collation comparisons are order of magnitude slower (~7-10x) than plain binary comparisons. Improve the performance by optimizing lowercase comparison function for UTF8String instances instead of performing full lowercase conversion before binary comparison.

Issue Links

• links to: GitHub Pull Request #45816
• links to: GitHub Pull Request #46283