[SPARK-35435] Not able to capture driver and executor logs in console

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.2.0
• Component/s: Spark Submit
• Labels: log-level

Environment

Production

Docs Text

Running the spark job using the below command, but not able to capture the driver and executor logs in console

# Run on a Spark standalone cluster in cluster deploy mode with supervise
./bin/spark-submit \ --class org.apache.spark.examples.SparkPi \ --master spark://207.184.161.138:7077 \ --deploy-mode cluster \ --supervise \ --executor-memory 20G \ --total-executor-cores 100 \ /path/to/examples.jar \ 1000

• Assignee: Unassigned
• Reporter: Sugumar
• Shepherd: Apache Spark
• Created: 18/May/21 14:06
• Updated: 12/Dec/22 18:11

Description

Running the spark job using the below command, but not able to capture the driver and executor logs in console
  # # Run on a Spark standalone cluster in cluster deploy mode with supervise

1. ./bin/spark-submit \ --class org.apache.spark.examples.SparkPi \ --master spark://207.184.161.138:7077 \ --deploy-mode cluster \ --supervise \ --executor-memory 20G \ --total-executor-cores 100 \ /path/to/examples.jar \ 1000