Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3129 Prevent data loss in Spark Streaming on driver failure using Write Ahead Logs
  3. SPARK-4029

Update streaming driver to reliably save and recover received block metadata on driver failures

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: DStreams
    • Labels:
      None
    • Target Version/s:

      Description

      As part of the initiative of preventing data loss on driver failure, this JIRA tracks the sub task of modifying the streaming driver to reliably save received block metadata, and recover them on driver restart.

        Attachments

          Activity

            People

            • Assignee:
              tdas Tathagata Das
              Reporter:
              tdas Tathagata Das
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: