Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-39589

Asynchronous I/O API support in Structured Streaming

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.3.0
    • None
    • Structured Streaming
    • None

    Description

      Often in streaming pipelines used to support ETL use cases, API calls to external systems can be found.  To better support and improve performance for such use cases, an asynchronous processing API should be added to Structured Streaming so that external requests can be handled in a more efficient manner.

      Attachments

        Issue Links

          Is contained by

          Umbrella - An overarching type made of sub-tasks SPARK-40025 Project Lightspeed: Faster and Simpler Stream Processing with Apache Spark

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              jerrypeng Boyang Jerry Peng
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: