[SPARK-4028] ReceivedBlockHandler interface to abstract the functionality of storage of received data

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Component/s: DStreams
• Target Version/s: 1.2.0
• Assignee: Tathagata Das
• Reporter: Tathagata Das
• Created: 21/Oct/14 02:14
• Updated: 30/Oct/14 21:52
• Resolved: 30/Oct/14 21:52

Description

As part of the initiative to prevent data loss on streaming driver failure, this JIRA tracks the subtask of implementing a ReceivedBlockHandler, that abstracts the functionality of storage of received data blocks. The default implementation will maintain the current behavior of storing the data into BlockManager. The optional implementation will store the data to both BlockManager as well as a write ahead log.

Issue Links

• links to: [Github] Pull Request #2940 (tdas)