Details
-
Improvement
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
3.1.1
-
None
Description
In a spark streaming application, there are 1000+ executors, more than 2000 events (executors events, job events) generated, then the jobs/job web page of spark is not able to show. The browser (chrome, firefox, safari) freezes. I had to open another window to open stages, executors pages by their addresses manually.
The jobs page is the home page, so this is rather annoyed. The problem is that the vis-timeline rending is too slow.
Their are some suggestion:
1) the page should not render timeline when the page is loading unless user clicks the link.
2) the executor group and job group should be separated, and user can choose to show one, both or neither.
3) the executor group should display executor event by time horizontally. Currently it is displayed executors one line by one line. If more than 100 executors, the page is not that good.
4)the vis-timeline library is not maintained anymore since 2017. Should be replaced a new one like https://github.com/visjs/vis-timeline
5) it is also good to get recent events, e.g. 500, and load more if user want see more, however the data could all be loaded once.
