[BEAM-2590] SparkRunner shim for Job API

• Type: Sub-task
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.13.0
• Component/s: runner-spark
• Labels: portability portability-spark
• Assignee: Kyle Weaver
• Reporter: Kenneth Knowles
• Created: 11/Jul/17 22:04
• Updated: 16/May/20 13:37
• Resolved: 23/Apr/19 00:40

Description

Whatever the result of https://s.apache.org/beam-job-api we will need a way for the JVM-based SparkRunner to receive and run pipelines authors in Python.

Issue Links

• is blocked by: BEAM-6685 JobServer driver with stubbed out spark related component
• is blocked by: BEAM-6686 Spark job invoker with stubbed out translator
• is blocked by: BEAM-6687 Spark Translator - GROUP_BY_KEY_TRANSFORM_URN
• is blocked by: BEAM-6688 Spark Translator - FLATTEN_TRANSFORM_URN
• is blocked by: BEAM-6689 Spark Translator - IMPULSE_TRANSFORM_URN
• is blocked by: BEAM-6690 Spark Translator - ASSIGN_WINDOWS_TRANSFORM_URN
• is blocked by: BEAM-6691 Spark Translator - ExecutableStage.URN
• is blocked by: BEAM-6692 Spark Translator - RESHUFFLE_URN
• is duplicated by: BEAM-7037 Spark portable runner: run wordcount.py
• links to: GitHub Pull Request #8115