Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-2358

zeppelin throw PermGen out of memory after a long time running

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: 0.6.2
    • Fix Version/s: None
    • Component/s: Interpreters
    • Labels:
    • Environment:

      Description

      1、setup zeppelin with ambari
      2、config interpreter with a greate mount of PermGen Memory, use share mode for spark interpreter
      3、create a notebook in zeppelin
      4、run spark code

      For some reason I need to use one notebook again and again.

      After running like 1000 lines code in notebook.
      Zeppelin throw like PermGen Space Out of Memory Exception.

      And notebook won't return from running any more.

      After check what's in memory with Jprofile, we see result as follow:JUST TOO MUCH LINES.......

      Would it be possible if we add some reset things for each interpreter, so we can reuse notebook with out delete and create then again?

      =================================================================================
      If I change interpreter mode to scope, and delete notebook when i finish some scala code compute. (Because I saw code scope mode will use different interpreter instance and the SparkIMain is not a static field)

      I found permgen space is not the problem anymore, but E , O, S0 is 100% taken. And notebook won't return from running any more.

      I also check what's in memory .. Fount out no more lines class as I expected, but with ZipArchive???

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              poseidon poseidon
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 12h
                12h
                Remaining:
                Remaining Estimate - 12h
                12h
                Logged:
                Time Spent - Not Specified
                Not Specified