[BEAM-7421] Flink/Spark non-portable runners doesn't override Reshuffle

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.14.0
• Component/s: runner-flink, runner-spark
• Assignee: Maximilian Michels
• Reporter: Mike Kaplinskiy
• Created: 24/May/19 20:05
• Updated: 16/May/20 13:36
• Resolved: 27/May/19 15:05

Description

Both the flink and the spark runners attempt to override the Reshuffle transform via the urn - e.g. https://github.com/apache/beam/blob/master/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkBatchTransformTranslators.java#L114 .

Unfortunately there's no TransformPayloadTranslatorRegistrar for Reshuffle. This means the call to urnForTransformOrNull will always return null for a Reshuffle so they are never translated.

 

Issue Links

• links to: GitHub Pull Request #8693