Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28594 Allow event logs for running streaming apps to be rolled over
  3. SPARK-28869

Roll over event log files

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • Spark Core
    • None

    Description

      This issue tracks the effort on rolling over event log files in driver and let SHS replay the multiple event log files correctly.

      This issue doesn't deal with overall size of event log, as well as no guarantee when deleting old event log files.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue SPARK-45771 Enable spark.eventLog.rolling.enabled by default

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #25670

          Web Link GitHub Pull Request #27576

          Activity

            People

              kabhwan Jungtaek Lim
              kabhwan Jungtaek Lim
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: