[HIVE-10179] Optimization for SIMD instructions in Hive

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: optimization
• Assignee: Chengxiang Li
• Reporter: Chengxiang Li
• Created: 01/Apr/15 07:59
• Updated: 04/Mar/18 11:22

Description

SIMD instuctions could be found in most of current CPUs, such as Intel's SSE2, SSE3, SSE4.x, AVX and AVX2, and it would help Hive to outperform if we can vectorize the mathematical manipulation part of Hive. This umbrella JIRA may contains but not limited to the subtasks like:

1. Code schema adaption, current JVM is quite strictly on the code schema which could be transformed into SIMD instructions during execution.
2. New implementation of mathematical manipulation part of Hive which designed to be optimized for SIMD instructions.

Issue Links

• is related to: HIVE-4160 Vectorized Query Execution in Hive

Sub-Tasks

• Loop optimization for SIMD in ColumnArithmeticColumn.txt: Closed
• Create a micro benchmark tool for vectorization to evaluate the performance gain after SIMD optimization: Resolved
• Loop optimization for SIMD in ColumnDivideColumn.txt: Closed
• Loop optimization for SIMD in IfExprColumnColumn.txt: Closed
• Loop optimization for SIMD in logical operators: Closed
• Loop optimization for SIMD in IfExprLong*Long*: Patch Available
• Loop optimization for SIMD in integer comparisons: Closed
• Byte array comparison optimization for SIMD: Resolved
• Boolean bitwise operation optimization for SIMD: Open
• Categorize vectorization benchmarks into arithmetic, comparison, logic: Closed
• Loop optimization for SIMD in IfExprDouble*Double*: Open
• Loop optimization for SIMD in double comparisons: Open