[HIVE-17043] Remove non unique columns from group by keys if not referenced later

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Logical Optimizer
• Assignee: Vineet Garg
• Reporter: Ashutosh Chauhan
• Created: 05/Jul/17 12:56
• Updated: 27/Feb/24 22:23
• Resolved: 15/Oct/18 03:35

Description

Group by keys may be a mix of unique (or primary) keys and regular columns. In such cases presence of regular column won't alter cardinality of groups. So, if regular columns are not referenced later, they can be dropped from group by keys. Depending on operator tree may result in those columns not being read at all from disk in best case. In worst case, we will avoid shuffling and sorting regular columns from mapper to reducer, which still could be substantial CPU and network savings.

Issue Links

• duplicates: HIVE-17041 Aggregate elimination with UNIQUE and NOT NULL column
• duplicates: HIVE-17042 Expose NOT NULL constraint in optimizer so constant folding can take advantage of it
• links to: ReivewBoard