[SPARK-13006] Use Log4j for Spark Standalone Executor Logging

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Component/s: Spark Core
• Labels: bulk-closed executors logging
• Assignee: Unassigned
• Reporter: Alan Braithwaite
• Created: 26/Jan/16 19:15
• Updated: 21/May/19 04:35
• Resolved: 21/May/19 04:35

Description

Currently it logs to scratch space which is shared with things like the application jar. It makes it very difficult to collect these logs effectively and cleanly.

https://github.com/apache/spark/blob/08c781ca672820be9ba32838bbe40d2643c4bde4/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala#L166-L172