[SPARK-26539] Remove spark.memory.useLegacyMode memory settings + StaticMemoryManager

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Spark Core
• Labels: release-notes
• Target Version/s: 3.0.0

Docs Text

The legacy memory manager, enabled by spark.memory.useLegacyMode, has been removed in Spark 3. It is effectively always false, its default since Spark 1.6. Associated settings spark.shuffle.memoryFraction, spark.storage.memoryFraction,and spark.storage.unrollFraction have been removed as well.

• Assignee: Sean R. Owen
• Reporter: Sean R. Owen
• Created: 04/Jan/19 21:20
• Updated: 08/Feb/22 21:37
• Resolved: 10/Jan/19 14:57

Description

The old memory manager was superseded by UnifiedMemoryManager in Spark 1.6, and has been the default since. I think we could remove it to simplify the code a little, but more importantly to reduce the variety of memory settings users are confronted with when using Spark.

Issue Links

• is related to: SPARK-38144 Remove unused `spark.storage.safetyFraction` config
• links to: GitHub Pull Request #23457