Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8834

Throttle DStreams dynamically through back-pressure information

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • None
    • None
    • DStreams
    • None

    Description

      This aims to have Spark Streaming be more resilient to high-throughput situations through back-pressure signaling & dynamic throttling.

      The Design doc can be found there:
      https://issues.apache.org/jira/browse/SPARK-8834
      An (outdated) PoC implementation exists.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              huitseeker Fran├žois Garillot
              Tathagata Das Tathagata Das
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: