[HIVE-27269] VectorizedMapJoin returns wrong result for TPC-DS query 97

• Type: Sub-task
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 4.0.0
• Labels: hive-4.0.0-must pull-request-available
• Assignee: Seonggon Namgung
• Reporter: Seonggon Namgung
• Created: 19/Apr/23 06:03
• Updated: 29/Mar/24 17:49
• Resolved: 30/Nov/23 12:37

Description

TPC-DS query 97 returns wrong results when hive.auto.convert.join and hive.vectorized.execution.enabled are set to true.

 

Result of query 97 on 1TB text dataset:

CommonMergeJoinOperator(hive.auto.convert.join=false): 534151529, 284185746, 84163

MapJoinOperator(hive.auto.convert.join=true, hive.vectorized.execution.enabled=false): 534151529, 284185746, 84163

VectorMapJoinOperator(hive.auto.convert.join=true, hive.vectorized.execution.enabled=true): 534151529, 284185388, 84163

 

Also I observed that VectorizedMapJoin returns different results for 100GB dataset when I run query 97 twice, but I could not reproduce it since then.

Issue Links

• is caused by: HIVE-25583 Support parallel load for HastTables - Interfaces
• links to: GitHub Pull Request #4857