[HIVE-3996] Correctly enforce the memory limit on the multi-table map-join

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11.0
• Fix Version/s: 0.11.0
• Component/s: Query Processor
• Hadoop Flags: Reviewed
• Tags: HIVE-3784
• Assignee: Vikram Dixit K
• Reporter: Vikram Dixit K
• Created: 07/Feb/13 02:50
• Updated: 03/Feb/18 20:13
• Resolved: 16/Apr/13 07:22

Description

Currently with HIVE-3784, the joins are converted to map-joins based on checks of the table size against the config variable: hive.auto.convert.join.noconditionaltask.size.

However, the current implementation will also merge multiple mapjoin operators into a single task regardless of whether the sum of the table sizes will exceed the configured value.

Issue Links

• blocks: HIVE-4071 Map-join outer join produces incorrect results.
• blocks: HIVE-4078 Delay the serialize-deserialize pair in CommonJoinTaskDispatcher
• is blocked by: HIVE-3891 physical optimizer changes for auto sort-merge join