Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-39973

Avoid noisy warnings logs when spark.scheduler.listenerbus.metrics.maxListenerClassesTimed = 0

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • Spark Core
    • None

    Description

      If spark.scheduler.listenerbus.metrics.maxListenerClassesTimed has been set to 0 to disable listener timers then listener registration will trigger noisy warnings like

      LiveListenerBusMetrics: Not measuring processing time for listener class org.apache.spark.sql.util.ExecutionListenerBus because a maximum of 0 listener classes are already timed.

      warnings.

      We should change the code to not print this warning when maxListenerClassesTimed = 0.

      I don't plan to work on this myself.

      Attachments

        Activity

          People

            gurwls223 Hyukjin Kwon
            joshrosen Josh Rosen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: