Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19633

FileSource read from FileSink

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 2.1.0
    • 2.2.0
    • Structured Streaming
    • None

    Description

      Right now, you can't start a streaming query from a location that is being written to by the file sink.

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. SPARK-25331 Structured Streaming File Sink duplicates records in case of driver failure

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #16987 [Github] Pull Request #16987 (lw-lin)

          Activity

            People

              lwlin Liwei Lin
              marmbrus Michael Armbrust
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: