[SPARK-48124] Disable structured logging for Interpreter by default

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Spark Core
• Labels: pull-request-available
• Assignee: Gengliang Wang
• Reporter: Gengliang Wang
• Created: 04/May/24 01:05
• Updated: 05/May/24 05:47
• Resolved: 04/May/24 18:48

Description

Since there are plain text output from Interpreters(spark-shell/spark-sql/pyspark), it makes more sense to disable structured logging for Interpreters by default.

 

spark-shell output when with structured logging enabled:

```

Setting default log level to "WARN".

To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).

Welcome to

      ____              __

     / _/  ___ _____/ /_

    \ \/ _ \/ _ `/ __/  '/

   /__/ ./_,// //_\   version 4.0.0-SNAPSHOT

      /_/

         

Using Scala version 2.13.13 (OpenJDK 64-Bit Server VM, Java 17.0.9)

Type in expressions to have them evaluated.

Type :help for more information.

{"ts":"2024-05-04T01:11:03.797Z","level":"WARN","msg":"Unable to load native-hadoop library for your platform... using builtin-java classes where applicable","logger":"NativeCodeLoader"} {"ts":"2024-05-04T01:11:04.104Z","level":"WARN","msg":"Service 'SparkUI' could not bind on port 4040. Attempting port 4041.","logger":"Utils"}

Spark context Web UI available at http://10.10.114.155:4041

Spark context available as 'sc' (master = local[*], app id = local-1714785064155).

Spark session available as 'spark'.

```

 

spark-shell output when without structured logging enabled:

```

Setting default log level to "WARN".

To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).

Welcome to

      ____              __

     / _/  ___ _____/ /_

    \ \/ _ \/ _ `/ __/  '/

   /__/ ./_,// //_\   version 4.0.0-SNAPSHOT

      /_/

         

Using Scala version 2.13.13 (OpenJDK 64-Bit Server VM, Java 17.0.9)

Type in expressions to have them evaluated.

Type :help for more information.

24/05/03 18:11:35 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

24/05/03 18:11:35 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.

Spark context Web UI available at http://10.10.114.155:4041

Spark context available as 'sc' (master = local[*], app id = local-1714785095892).

Spark session available as 'spark'.

```

Issue Links

• links to: GitHub Pull Request #46383
• links to: GitHub Pull Request #46387