Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25186 Stabilize Data Source V2 API
  3. SPARK-25187

Revisit the life cycle of ReadSupport instances.

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.4.0
    • None
    • SQL

    Description

      Currently the life cycle is bound to the batch/stream query. This fits streaming very well but may not be perfect for batch source. We can also consider to let ReadSupport.newScanConfigBuilder take DataSourceOptions as parameter, if we decide to change the life cycle.

      Attachments

        Activity

          People

            Unassigned Unassigned
            cloud_fan Wenchen Fan
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: