Description
Memory leak on driver for a long time running application. We have application using structured streaming and running 48 hours. But driver fails with out of memory after 25 hours. After investigating heap dump we found that most of the memory was occupied with a lot of SparkPlanGraphWrapper objects inside InMemoryStore.
Application was run with option:
--driver-memory 4G