[SPARK-23411] Deprecate SparkContext.getRDDStorageInfo

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: Marcelo Masiero Vanzin
• Created: 13/Feb/18 14:52
• Updated: 16/Mar/20 22:52

Description

This is another developer API in SparkContext that is better moved to a more monitoring-minded place such as SparkStatusTracker. The same information is also already available through the REST API.

Moreover, it exposes RDDInfo, which is a mutable internal Spark type, and exposing that to applications is kinda sketchy. (It can't be made private, though, since it's exposed indirectly in events posted to the listener bus.)