Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31593

Remove unnecessary streaming query progress update

    XMLWordPrintableJSON

Details

    • Bug
    • Status: In Progress
    • Minor
    • Resolution: Unresolved
    • 2.4.5, 3.0.0
    • None
    • Structured Streaming
    • None

    Description

      Structured Streaming progress reporter will always report an `empty` progress when there is no new data. As design, we should provide progress updates every 10s (default) when there is no new data.

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. SPARK-31928 Flaky test: StreamingDeduplicationSuite.test no-data flag

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #28391 [Github] Pull Request #28391 (uncleGen)

          Activity

            People

              Unassigned Unassigned
              uncleGen Genmao Yu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: