Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-9826

Cannot use custom classes in log4j.properties

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.1
    • Fix Version/s: 1.4.2, 1.5.0
    • Component/s: Spark Core
    • Labels:

      Description

      log4j is initialized before spark class loader is set on the thread context.
      Therefore it cannot use classes embedded in fat-jars submitted to spark.

      While parsing arguments, spark calls methods on Utils class and triggers ShutdownHookManager static initialization. This then leads to log4j being initialized before spark gets the chance to specify custom class MutableURLClassLoader on the thread context.

      See detailed explanation here:
      http://apache-spark-user-list.1001560.n3.nabble.com/log4j-custom-appender-ClassNotFoundException-with-spark-1-4-1-tt24159.html

        Attachments

          Activity

            People

            • Assignee:
              FlamingMike Michel Lemay
              Reporter:
              FlamingMike Michel Lemay
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: