Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16416

Logging in shutdown hook does not work properly with Log4j 2.x

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.6.2
    • Fix Version/s: 2.1.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      Spark registers some shutdown hooks, and they log messages during shutdown:

      https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala#L58

      Since the Logging trait creates SLF4J loggers lazily:
      https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/internal/Logging.scala#L47

      a SLF4J logger is created during the execution of the shutdown hook.

      This does not work when Log4j 2.x is used as SLF4J implementation:
      https://issues.apache.org/jira/browse/LOG4J2-1222

      Even though Log4j 2.6 handles this more gracefully than before, it still does emit a warning and will not be able to process the log message properly.

      Proposed solution: make sure to eagerly create the SLF4J logger to be used in shutdown hooks when registering the hook.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mikaelstaldal Mikael Ståldal
                Reporter:
                mikaelstaldal Mikael Ståldal
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: