Description
Try run
sc.parallelize(1 to 100, 1000000).cache().count()
And open the storage UI for this RDD. It takes forever to load the page.
When the number of partitions is very large, I think there are a few alternatives:
0. Only show the top 1000.
1. Pagination
2. Instead of grouping by RDD blocks, group by executors