Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3129 Prevent data loss in Spark Streaming on driver failure using Write Ahead Logs
  3. SPARK-4029

Update streaming driver to reliably save and recover received block metadata on driver failures

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 1.2.0
    • DStreams
    • None

    Description

      As part of the initiative of preventing data loss on driver failure, this JIRA tracks the sub task of modifying the streaming driver to reliably save received block metadata, and recover them on driver restart.

      Attachments

        Activity

          People

            tdas Tathagata Das
            tdas Tathagata Das
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: