Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-701 Change new Java API functions to SAMs
  3. FLINK-848

Move combine() from GroupReduceFunction to Interface

    XMLWordPrintableJSON

Details

    Description

      Currently, the combine method of the GroupReduceFunction allows to return multiple values using a collector. However, most combiners do not need this because they return only a single value. Furthermore, a single value returning combiner can be executed using more efficient hash-based strategies.

      Hence, we propose to introduce a combine method for GroupReduce which returns only a single value. In order to keep support for the rare cases where more than one value needs to be returned, we want to keep the collector-combiner as well.

      To do so, we could remove the combine method from the abstract GroupReduceFunction class and add two Combinable interfaces, one for a single-value and one for a multi-value combiner.
      This would also make the Combinable annotation obsolete as the optimizer can check whether a GroupReduceFunction implements one of the Combinable interfaces or not.

      ---------------- Imported from GitHub ----------------
      Url: https://github.com/stratosphere/stratosphere/issues/848
      Created by: fhueske
      Labels: core, enhancement,
      Milestone: Release 0.6 (unplanned)
      Created at: Thu May 22 10:23:04 CEST 2014
      State: open

      Attachments

        Activity

          People

            ktzoumas Kostas Tzoumas
            fhueske Fabian Hueske
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: