Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
4.0.0
Description
Since there are plain text output from Interpreters(spark-shell/spark-sql/pyspark), it makes more sense to disable structured logging for Interpreters by default.
spark-shell output when with structured logging enabled:
```
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Welcome to
____ __
/ _/ ___ _____/ /_
\ \/ _ \/ _ `/ __/ '/
/__/ ./_,// //_\ version 4.0.0-SNAPSHOT
/_/
Using Scala version 2.13.13 (OpenJDK 64-Bit Server VM, Java 17.0.9)
Type in expressions to have them evaluated.
Type :help for more information.
{"ts":"2024-05-04T01:11:03.797Z","level":"WARN","msg":"Unable to load native-hadoop library for your platform... using builtin-java classes where applicable","logger":"NativeCodeLoader"} {"ts":"2024-05-04T01:11:04.104Z","level":"WARN","msg":"Service 'SparkUI' could not bind on port 4040. Attempting port 4041.","logger":"Utils"}Spark context Web UI available at http://10.10.114.155:4041
Spark context available as 'sc' (master = local[*], app id = local-1714785064155).
Spark session available as 'spark'.
```
spark-shell output when without structured logging enabled:
```
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Welcome to
____ __
/ _/ ___ _____/ /_
\ \/ _ \/ _ `/ __/ '/
/__/ ./_,// //_\ version 4.0.0-SNAPSHOT
/_/
Using Scala version 2.13.13 (OpenJDK 64-Bit Server VM, Java 17.0.9)
Type in expressions to have them evaluated.
Type :help for more information.
24/05/03 18:11:35 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
24/05/03 18:11:35 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
Spark context Web UI available at http://10.10.114.155:4041
Spark context available as 'sc' (master = local[*], app id = local-1714785095892).
Spark session available as 'spark'.
```