[IGNITE-9357] Spark Structured Streaming with Ignite as data source and sink

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0
• Component/s: spark
• Ignite Flags: Docs Required
• Assignee: Unassigned
• Reporter: Alexey Kukushkin
• Created: 23/Aug/18 09:46
• Updated: 18/Jun/24 21:14

Description

We are working on a PoC where we want to use Ignite as a data storage and Spark as a computation engine. We found that Ignite is supported neither as a source nor as a Sink when using Spark Structured Streaming, which is a must for us.

We are enhancing Ignite to support Spark streaming with Ignite. We will send docs and code for review for the Ignite Community to consider if the community wants to accept this feature. 

Issue Links

• links to: GitHub Pull Request #4603
• links to: GitHub Pull Request #4603