[BEAM-2453] The Java DirectRunner should exercise all parts of a CombineFn

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.2.0
• Component/s: runner-direct
• Assignee: Thomas Groh
• Reporter: Thomas Groh
• Created: 15/Jun/17 22:17
• Updated: 16/May/20 13:46
• Resolved: 28/Jul/17 22:26

Description

Specifically it should:

Create some number of accumulators; add elements to these accumulators, merge the created accumulators, and extract the output.

This can be performed by replacing the Combine.perKey composite transform with a multi-step combine CombineBundles -> GroupByKey -> MergeAccumulators

Where CombineBundles is a ParDo which takes input KV<K, InputT> and produces KV<K, AccumT>, outputting in FinishBundle (this can only be performed if the Combine takes no side inputs or does not have merging windows). MergeAccumulators takes in KV<K, Iterable<AccumT>> and produces KV<K, OutputT> by merging all of the accumulators and extracting the output.

Issue Links

• links to: GitHub Pull Request #3579