Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30656

Support the "minPartitions" option in Kafka batch source and streaming source v1

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: Structured Streaming
    • Labels:
      None

      Description

      Right now, the "minPartitions" option only works in Kafka streaming source v2. It would be great that we can support it in batch and streaming source v1 (v1 is the fallback mode when a user hits a regression in v2) as well.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zsxwing Shixiong Zhu
                Reporter:
                zsxwing Shixiong Zhu
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: