[HIVE-16328] HoS: more aggressive mapjoin optimization when hive.spark.use.ts.stats.for.mapjoin is true

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Logical Optimizer
• Assignee: Chao Sun
• Reporter: Chao Sun
• Created: 29/Mar/17 21:56
• Updated: 22/May/18 23:59
• Resolved: 12/Apr/17 20:41

Description

In HIVE-15489, when hive.spark.use.ts.stats.for.mapjoin is set to true, and if the JOIN op has any upstream RS operator, then we will stop converting the JOIN op to MAPJOIN op.

However, this is overly conservative. A better solution is to treat the branch that has upstream RS as the big table and check if all other branches are map-only AND can fit in hash table size.

Issue Links

• is related to: HIVE-15489 Alternatively use table scan stats for HoS