[HIVE-14707] ACID: Insert shuffle sort-merges on blank KEY

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.3.0
• Component/s: Transactions
• Assignee: Eugene Koifman
• Reporter: Gopal Vijayaraghavan
• Created: 06/Sep/16 17:29
• Updated: 21/Jul/17 18:12
• Resolved: 22/Jan/17 00:31

Description

The ACID insert codepath uses a sorted shuffle, while they key used for shuffle is always 0 bytes long.

hive (sales_acid)> explain insert into sales values(1, 2, '3400-0000-0000-009', 1, null);

STAGE PLANS:
  Stage: Stage-1
    Tez
      DagId: gopal_20160906172626_80261c4c-79cc-4e02-87fe-3133be404e55:2
      Edges:
        Reducer 2 <- Map 1 (SIMPLE_EDGE)
...
      Vertices:
        Map 1 
            Map Operator Tree:
                TableScan
                  alias: values__tmp__table__2
                  Statistics: Num rows: 1 Data size: 28 Basic stats: COMPLETE Column stats: NONE
                  Select Operator
                    expressions: tmp_values_col1 (type: string), tmp_values_col2 (type: string), tmp_values_col3 (type: string), tmp_values_col4 (type: string), tmp_values_col5 (type: string)
                    outputColumnNames: _col0, _col1, _col2, _col3, _col4
                    Statistics: Num rows: 1 Data size: 28 Basic stats: COMPLETE Column stats: NONE
                    Reduce Output Operator
                      sort order: 
                      Map-reduce partition columns: UDFToLong(_col1) (type: bigint)
                      Statistics: Num rows: 1 Data size: 28 Basic stats: COMPLETE Column stats: NONE
                      value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string)
            Execution mode: vectorized, llap
            LLAP IO: no inputs

Note the missing "+" / "-" in the Sort Order fields.

Issue Links

• Dependent: HIVE-15573 Vectorization: Non-Uniform shuffle ReduceSink is not specialized
• is related to: HIVE-11858 Hive DISTRIBUTE BY & CROSS PRODUCT should not sort output