Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Currently, the combine method of the GroupReduceFunction allows to return multiple values using a collector. However, most combiners do not need this because they return only a single value. Furthermore, a single value returning combiner can be executed using more efficient hash-based strategies.
Hence, we propose to introduce a combine method for GroupReduce which returns only a single value. In order to keep support for the rare cases where more than one value needs to be returned, we want to keep the collector-combiner as well.
To do so, we could remove the combine method from the abstract GroupReduceFunction class and add two Combinable interfaces, one for a single-value and one for a multi-value combiner.
This would also make the Combinable annotation obsolete as the optimizer can check whether a GroupReduceFunction implements one of the Combinable interfaces or not.
---------------- Imported from GitHub ----------------
Url: https://github.com/stratosphere/stratosphere/issues/848
Created by: fhueske
Labels: core, enhancement,
Milestone: Release 0.6 (unplanned)
Created at: Thu May 22 10:23:04 CEST 2014
State: open