[BEAM-10789] Add support for checkpointing in Spark streaming

• Type: Improvement
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: runner-spark
• Labels: portability-spark
• Assignee: Unassigned
• Reporter: Anna Qin
• Created: 21/Aug/20 17:45
• Updated: 04/Jun/22 17:17

Description

Spark streaming (both portable and non-portable) currently uses queueStream to create the initial DStream from a queue of RDDs. However, queueStream does not support checkpointing.