[SPARK-10420] Implementing Reactive Streams based Spark Streaming Receiver

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Incomplete
• Component/s: DStreams
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Nilanjan Raychaudhuri
• Shepherd: Tathagata Das
• Created: 02/Sep/15 17:16
• Updated: 21/May/19 04:36
• Resolved: 21/May/19 04:36

Description

Hello TD,

This is probably the last bit of the back-pressure story, implementing ReactiveStreams based Spark streaming receivers. After discussing about this with my Typesafe team we came up with the following design document

https://docs.google.com/document/d/1lGQKXfNznd5SPuQigvCdLsudl-gcvWKuHWr0Bpn3y30/edit?usp=sharing

Could you please take a look at this when you get a chance?

Thanks
Nilanjan

Issue Links

• is related to: SPARK-7398 Add back-pressure to Spark Streaming (umbrella JIRA)