Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23670

Memory leak of SparkPlanGraphWrapper in sparkUI

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.1, 2.4.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Memory leak on driver for a long time running application. We have application using structured streaming and running 48 hours. But driver fails with out of memory after 25 hours. After investigating heap dump we found that most of the memory was occupied with a lot of SparkPlanGraphWrapper objects inside InMemoryStore.

       Application was run with option: 
      --driver-memory 4G

        Attachments

        1. heap.png
          99 kB
          Myroslav Lisniak
        2. heapdump_OOM.png
          994 kB
          Michael Spector
        3. Screen Shot 2018-09-06 at 23.19.56.png
          318 kB
          Michael Spector

          Activity

            People

            • Assignee:
              myroslav.lisniak Myroslav Lisniak
              Reporter:
              myroslav.lisniak Myroslav Lisniak

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment