[SPARK-29097] Spark driver memory exceeded the storage memory

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Incomplete
• Affects Version/s: 2.3.3, 2.4.4
• Component/s: Block Manager, Spark Core, Web UI
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: George Papa
• Created: 16/Sep/19 13:06
• Updated: 25/May/21 01:49
• Resolved: 25/May/21 01:42

Description

In Spark UI, the driver used storage memory (2.1GB) exceeds the total available memory (1.5GB). When I use the same application with Spark 2.1.1 I don't have this behavior.