[HIVE-13872] Vectorization: Fix cross-product reduce sink serialization

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 2.2.0
• Component/s: Vectorization
• Target Version/s: 2.2.0
• Assignee: Matt McCline
• Reporter: Gopal Vijayaraghavan
• Created: 27/May/16 03:57
• Updated: 27/Feb/24 22:23
• Resolved: 23/Jun/16 10:42

Description

TPC-DS Q13 produces a cross-product without CBO simplifying the query

Caused by: java.lang.RuntimeException: null STRING entry: batchIndex 0 projection column num 1
        at org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.nullBytesReadError(VectorExtractRow.java:349)
        at org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRowColumn(VectorExtractRow.java:267)
        at org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRow(VectorExtractRow.java:343)
        at org.apache.hadoop.hive.ql.exec.vector.VectorReduceSinkOperator.process(VectorReduceSinkOperator.java:103)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
        at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
        at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:762)
        ... 18 more

Simplified query

set hive.cbo.enable=false;

-- explain

select count(1)  
 from store_sales
     ,customer_demographics
 where (
( 
  customer_demographics.cd_demo_sk = store_sales.ss_cdemo_sk
  and customer_demographics.cd_marital_status = 'M'
     )or
     (
   customer_demographics.cd_demo_sk = ss_cdemo_sk
  and customer_demographics.cd_marital_status = 'U'
     ))
;

        Map 3 
            Map Operator Tree:
                TableScan
                  alias: customer_demographics
                  Statistics: Num rows: 1920800 Data size: 717255532 Basic stats: COMPLETE Column stats: NONE
                  Reduce Output Operator
                    sort order: 
                    Statistics: Num rows: 1920800 Data size: 717255532 Basic stats: COMPLETE Column stats: NONE
                    value expressions: cd_demo_sk (type: int), cd_marital_status (type: string)
            Execution mode: vectorized, llap
            LLAP IO: all inputs

Issue Links

• links to: RB #48949