Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-36756

Spark uncaught exception handler is using logError

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.1.0, 3.1.1, 3.1.2, 3.2.0, 3.0.4
    • None
    • Spark Core
    • None

    Description

      Spark is setting up an handler to catch any uncaught exception.

      This handler itself catch any subsequent exception that can happen while reporting the initially uncaught exception.

      Issue is that if the subsequent exception is due to a log4j issue, as the catch there is also using logError, it will loop, not display any exception in the logs, and exiting with code 51.

      e.g. We got an log4j issue when using logstash JSON logging :

      at net.logstash.log4j.JSONEventLayoutV1.format(JSONEventLayoutV1.java:137)
      at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:310)
      at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
      at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
      at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
      at org.apache.log4j.Category.callAppenders(Category.java:206)
      at org.apache.log4j.Category.forcedLog(Category.java:391)
      at org.apache.log4j.Category.log(Category.java:856)
      at org.slf4j.impl.Log4jLoggerAdapter.error(Log4jLoggerAdapter.java:576)
      at org.apache.spark.internal.Logging.logError(Logging.scala:94)
      at org.apache.spark.internal.Logging.logError$(Logging.scala:93)
      at org.apache.spark.util.SparkUncaughtExceptionHandler.logError(SparkUncaughtExceptionHandler.scala:28)
      at org.apache.spark.util.SparkUncaughtExceptionHandler.uncaughtException(SparkUncaughtExceptionHandler.scala:37)
      at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:1057)
      at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:1052)
      at java.lang.Thread.dispatchUncaughtException(Thread.java:1959)
      

      Suggested fix:

      Directly using println and printStackTrace as safe fallback in this catch.

      Attachments

        Activity

          People

            Unassigned Unassigned
            cchantepie C├ędric Chantepie
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: