[HIVE-15741] Faster unsafe byte array comparisons

• Type: Improvement
• Status: Patch Available
• Priority: Minor
• Resolution: Unresolved
• Assignee: Teddy Choi
• Reporter: Teddy Choi
• Created: 27/Jan/17 01:15
• Updated: 05/Feb/17 13:42

Description

Byte array comparison is heavily used in joins and string conditions. Pure Java implementation is simple but not performant. An implementation with Unsafe#getLong is much faster. It's already implemented in org.apache.hadoop.io.WritableComparator#compare. The WritableComparator class handles exceptional cases, including a different endian and no access to Unsafe, and it was used for many years in production.

This patch will replace pure Java byte array comparisons with safe and faster unsafe ones to get more performance.

Issue Links

• duplicates: HIVE-12348 Byte array comparison optimization for SIMD
• links to: Review Board