[SPARK-31971] Add pagination support for all jobs timeline

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.1, 3.1.0
• Component/s: Web UI
• Assignee: Kousuke Saruta
• Reporter: Kousuke Saruta
• Created: 11/Jun/20 20:26
• Updated: 11/Jun/20 20:52

Description

If there are lots of jobs, rendering performance of all jobs timeline can significantly goes down. This issue is reported in SPARK-31967.
For example, the following operation can take >40 sec.

(1 to 1000).foreach(_ => sc.parallelize(1 to 10).collect) 

Although it's not the fundamental solution, pagination can mitigate the issue.

Issue Links

• is related to: SPARK-31967 Loading jobs UI page takes 40 seconds
• links to: [Github] Pull Request #28803 (sarutak)