Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-4131

Broadcaster memory leak

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v3.0.0, v2.6.4
    • Component/s: Metadata
    • Labels:
      None

      Description

      Test evidence

      Use MAT(Memory Analyzer) to analyze this issue.

      1. start  Kylin, dump a jvm heap info snapshot and import into MAT. We can see there is only one Broadcaster and ProjectManager object.
      1. Call 'list cube' api and 'reload metadata' api many times. We can see there are many BroadCaster and Manager objects in the memory, which should be recycled by GC, but they are not.

       

      Root cause

      Broadcaster hold the ref of Listener object defined in Manager, and the Listener object hold the ref of Manager, because the impl class of Listener is an inner class of Manager.

      Broadcaster.java

      private Map<String, List<Listener>> listenerMap = Maps.newConcurrentMap();
      

       ProjectManager.java -> ProjectSyncListener

      private class ProjectSyncListener extends Broadcaster.Listener {
      
      @Override
      public void onEntityChange(Broadcaster broadcaster, String entity, Event event, String cacheKey)
      throws IOException {
      String project = cacheKey;
      
      if (event == Event.DROP) {
      removeProjectLocal(project);
      return;
      }
      
      reloadProjectQuietly(project);
      broadcaster.notifyProjectSchemaUpdate(project);
      broadcaster.notifyProjectDataUpdate(project);
      }
      }
      
      

      And Broadcaster can't be recycled by GC because it holds a running thread, which cause the Manager object also not being recycled.

      After Fix

       The Broadcaster objects and Manager objects are recycled by GC.

       

        Attachments

        1. image-2019-08-12-17-22-23-477.png
          291 kB
          Chao Long
        2. image-2019-08-12-17-22-15-783.png
          271 kB
          Chao Long
        3. image-2019-08-12-17-16-05-532.png
          111 kB
          Chao Long
        4. image-2019-08-12-16-39-52-327.png
          291 kB
          Chao Long
        5. image-2019-08-12-16-39-46-262.png
          271 kB
          Chao Long

          Issue Links

            Activity

              People

              • Assignee:
                Wayne0101 Chao Long
                Reporter:
                Wayne0101 Chao Long
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: