[SPARK-23051] job description in Spark UI is broken

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.3.0
• Component/s: Web UI
• Labels: regression
• Target Version/s: 2.3.0
• Assignee: Sandor Murakozi
• Reporter: Shixiong Zhu
• Created: 12/Jan/18 01:16
• Updated: 20/Feb/18 19:58
• Resolved: 14/Jan/18 14:33

Description

In previous versions, Spark UI will use the stage description if the job description is not set. But right now it’s just empty.

Reproducer: Just run the following codes in spark shell and check the UI:

val q = spark.readStream.format("rate").load().writeStream.format("console").start()
Thread.sleep(2000)
q.stop()

Issue Links

• causes: SPARK-23470 org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow
• links to: [Github] Pull Request #20251 (smurakozi)