Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20971

Purge the metadata log for FileStreamSource

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.1.1
    • None
    • Structured Streaming

    Description

      Currently FileStreamSource.commit is empty. We can delete unused metadata logs in this method to reduce the size of log files.

      Attachments

        Activity

          People

            Unassigned Unassigned
            zsxwing Shixiong Zhu
            Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: