In previous versions, Spark UI will use the stage description if the job description is not set. But right now it’s just empty.
Reproducer: Just run the following codes in spark shell and check the UI:
val q = spark.readStream.format("rate").load().writeStream.format("console").start() Thread.sleep(2000) q.stop()
- causes
-
SPARK-23470 org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow
-
- Resolved
-
- links to