[SPARK-31621] Spark Master UI Fails to load if application is waiting for workers to launch driver

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0, 3.1.0
• Component/s: Spark Core, Web UI
• Assignee: Akshat Bordia
• Reporter: Akshat Bordia
• Created: 01/May/20 11:03
• Updated: 05/May/20 14:00
• Resolved: 05/May/20 14:00

Description

Steps to Reproduce:

1. Run Spark Master and 1 Worker (Using local machine).
2. Submit an (long-running) application using spark-submit command: ./bin/spark-submit --master spark://127.0.0.1:7077  --deploy-mode cluster --class org.apache.spark.examples.SparkPi examples/target/original-spark-examples_2.12-3.1.0-SNAPSHOT.jar
3. Load Spark Master UI (http://localhost:8080/) which should launch successfully.
4. Submit one more application using the spark-submit command.
5. Load Spark Master UI again.

Expected Behavior: Spark Master UI should launch successfully.

Actual Behavior: Spark Master UI launch fails with HTTP Error 500 as shown in the below screenshot.

Issue Links

• links to: [Github] Pull Request #28429 (akshatb1)