If the table is sorted by a given key, we don't use that for group by. That can be very useful.
For eg: if T is sorted by column c1,
For select c1, aggr() from T group by c1
we always use a single map-reduce job. No hash table is needed on the mapper, since the data is sorted by c1 anyway.
This will reduce the memory pressure on the mapper and also remove overhead of maintaining the hash table.
|Status||Open [ 1 ]||Resolved [ 5 ]|
|Resolution||Fixed [ 1 ]|
|Summary||Sorted Group By||Optimize GROUP BY aggregations where key is a sorted/bucketed column|
|Status||Resolved [ 5 ]||Closed [ 6 ]|
|Transition||Time In Source Status||Execution Times||Last Executer||Last Execution Date|
|20d 8h 17m||1||Namit Jain||04/Dec/09 04:39|
|742d 19h 26m||1||Carl Steinbach||17/Dec/11 00:05|