[HIVE-24205] Optimise CuckooSetBytes

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Labels: pull-request-available
• Assignee: Mustafa İman
• Reporter: Rajesh Balamohan
• Created: 28/Sep/20 11:09
• Updated: 07/Feb/24 19:07
• Resolved: 06/Oct/20 04:09

Description

FilterStringColumnInList, StringColumnInList  etc use CuckooSetBytes for lookup.

One option to optimize would be to add boundary conditions on "length" with the min/max length stored in the hashes (ref: https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/CuckooSetBytes.java#L85) . This would significantly reduce the number of hash computation that needs to happen. E.g TPCH-Q12

Issue Links

• is related to: ORC-1610 Reduce the number of hash computation in CuckooSetBytes
• links to: GitHub Pull Request #1549