[SPARK-25814] spark driver runs out of memory on org.apache.spark.util.kvstore.InMemoryStore

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.2.2, 2.3.2
• Component/s: Spark Core
• Labels: driver memory-analysis memory-leak statestore
• Assignee: Unassigned
• Reporter: Ruslan Dautkhanov
• Created: 23/Oct/18 20:06
• Updated: 23/Oct/18 20:59

Description

 We're looking into issue when even huge spark driver memory gets eventually exhausted and GC makes driver stop responding.

Used JXRay.com tool and found that most of driver heap is used by 

 

org.apache.spark.status.AppStatusStore
  -> org.apache.spark.status.ElementTrackingStore
    -> org.apache.spark.util.kvstore.InMemoryStore
 

 

Is there is a way to tune this particular spark driver's memory region down?