Details
-
Improvement
-
Status: Patch Available
-
Minor
-
Resolution: Unresolved
-
0.13.0
-
None
-
None
Description
This may be controversial, so I'd like others' opinions on this.
It is not currently possible to disable the combiner and use map-side aggregation at the same time. This is a problematic since map-side aggregation effectively combines in the mapper, so running the combiner adds expensive combiner execution (combiner requires deserialization & reserialization) for little to no value.
PIG-2829 had a patch to disable the combiner when map-side aggregation is used (along with some other changes). This was never integrated because the map-side aggregation code was redone while this was in progress.