[SPARK-10881] Unable to use custom log4j appender in spark executor

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Incomplete
• Affects Version/s: 1.3.1
• Component/s: Spark Core
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: David Morávek
• Created: 30/Sep/15 12:05
• Updated: 21/May/19 04:15
• Resolved: 21/May/19 04:15

Description

In CoarseGrainedExecutorBackend, log4j is initialized, before userclasspath gets registered:

https://github.com/apache/spark/blob/v1.3.1/core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala#L126

In order to use custom appender, one has to distribute it using `spark-submit --files` and set it via spark.executor.extraClassPath