[SPARK-33981] SparkUI: Storage page is empty even if cached

• Type: Question
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.3
• Component/s: Web UI

Environment

spark 2.3.3

• Assignee: Unassigned
• Reporter: Maple
• Created: 04/Jan/21 07:56
• Updated: 12/Dec/22 18:10
• Resolved: 07/Jan/21 10:00

Description

scala> import org.apache.spark.storage.StorageLevel
import org.apache.spark.storage.StorageLevel

scala> val rdd = sc.parallelize(1 to 10, 1).persist(StorageLevel.MEMORY_ONLY_2)
rdd: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[0] at parallelize at <console>:25

scala> rdd.count
res0: Long = 10

but SparkUI storage page is empty