[HIVE-1772] optimize join followed by a groupby

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Component/s: Query Processor
• Assignee: Unassigned
• Reporter: Namit Jain
• Created: 05/Nov/10 23:29
• Updated: 18/Jul/13 14:57
• Resolved: 18/Jul/13 14:57

Description

explain SELECT x.key, count(1) FROM src1 x JOIN src y ON (x.key = y.key) group by x.key;

STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-2 depends on stages: Stage-1
Stage-0 is a root stage

The above query issues 2 map-reduce jobs.
The first MR job performs the join, whereas the second MR performs the group by.
Since the data is already sorted, the group by can be performed in the reducer of the join itself.

Issue Links

• relates to: HIVE-3667 Umbrella jira for Correlation Optimizer
• relates to: HIVE-2206 add a new optimizer for query correlation discovery and optimization
• relates to: HIVE-3430 group by followed by join with the same key should be optimized

Sub-Tasks

• optimize orderby followed by a groupby: Closed