Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Invalid
-
0.2.0
-
None
-
None
Description
There are some cases where a combiner optimization chosen by Pig may actually be slower than the non optimized version. For example, the use of combiner to address the issue reported in https://issues.apache.org/jira/browse/PIG-580 could result in slower execution IF the distinct on groups of values does not actually shrink those groups. This is however very data dependent and the user may know before hand that this might be the case and may wish to disable the use of the optimizer. Pig should enable an option to do so.