[SPARK-23481] The job page shows wrong stages when some of stages are evicted

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.3.0
• Component/s: Web UI
• Target Version/s: 2.3.0
• Assignee: Shixiong Zhu
• Reporter: Shixiong Zhu
• Created: 21/Feb/18 19:14
• Updated: 21/Feb/18 23:37
• Resolved: 21/Feb/18 23:37

Description

Run "bin/spark-shell --conf spark.ui.retainedJobs=10 --conf spark.ui.retainedStages=10", type the following codes and click the job 19 page, it will show wrong stage ids:

val rdd = sc.parallelize(0 to 100, 100).repartition(10).cache()

(1 to 20).foreach { i =>
   rdd.repartition(10).count()
}

Please see the attached screenshots.

Issue Links

• links to: [Github] Pull Request #20654 (zsxwing)