[BEAM-1623] Transform Reshuffle directly in Spark runner

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 0.6.0
• Component/s: runner-spark
• Assignee: Aviem Zur
• Reporter: Aviem Zur
• Created: 04/Mar/17 13:47
• Updated: 16/May/20 13:39
• Resolved: 06/Mar/17 09:50

Description

Transform Reshuffle directly in Spark runner.
Spark's repartition is logically equivalent to Reshuffle and will cause the required fusion break without incurring the overhead of groupByKey (In streaming pipelines this will use updateStateByKey) compared to a simple repartition.

Issue Links

• links to: GitHub Pull Request #2160