Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-1475

Drain event logging queue before stopping event logger

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 1.0.0
    • Spark Core
    • None

    Description

      When stopping SparkListenerBus, its event queue needs to be drained. And this needs to happen before event logger is stopped. Otherwise, any event still waiting to be processed in the queue may be lost and consequently event log file may be incomplete.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-1407 EventLogging to HDFS doesn't work properly on yarn

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-1132 Persisting Web UI through refactoring the SparkListener interface

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Activity

            People

              kzhang Kan Zhang
              kzhang Kan Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: