Uploaded image for project: 'Groovy'
  1. Groovy
  2. GROOVY-7109

Severe Memory Leak Happens when Script Executes

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.3.6
    • Fix Version/s: 2.4.8
    • Component/s: groovy-runtime
    • Labels:
      None

      Description

      Recently I found my website creates so many classes in permGen. In dump file I can see so many strange classes such as java_util_List$size, java_util_List$subList.

      I don't know what are these classes, so I just call them "method classes". The serious thing is that, each "method class" has many "anonymous classes", such as java_util_List$size$0, java_util_List$size$1, java_util_List$size$2 ... There are over 1,000 java_util_List$size, and over 60,000 class totally in permGen.

      Then I test the groovy script engine and find that when script is excuting, these strange method is created.
      Here is my test code.

      TodoJava.java
              ScriptEngineManager m = new ScriptEngineManager();
              ScriptEngine engine = m.getEngineByName("groovy");
              CompiledScript script = ((Compilable) engine).compile(new FileReader( "test-script.groovy" ));
      

      The test-script.groovy file:

      test-script.groovy
      import cn.ming.youxi.TestClazz
      
      Random random = new Random();
      for (i in 1 .. 10) {
          random.nextInt(i);
      }
      TestClazz testClazz = clazz;
      List list = Arrays.asList(
              testClazz.att,
              testClazz.att,
              testClazz.att,
              testClazz.att,
              testClazz.att,
              testClazz.att
      );
      
      return list.get(2);
      

      TestClazz is a simple class with an Integer attribute. When I call this method, the "method classes" is created.

      TodoJava.java
              public void run(CompiledScript script) {
                  TestClazz testClazz = new TestClazz(new Random().nextInt());
                  Map<String, Object> params = new HashMap<String, Object>();
                  params.put("clazz", testClazz);
                  Bindings bindings = engine.createBindings();
                  bindings.putAll(params);
                  try {
                      System.out.println(script.eval(bindings));
                  } catch (ScriptException e) {
                      e.printStackTrace();
                  }
              }
      

      But there is only one java_util_List$get and java_util_Random$nextInt no matter how many times I call it. Then I try to execute the script in muti-thread. Here's the code:

      TodoJava.java
          static class MyThread extends Thread {
              ScriptEngine engine;
              CompiledScript script;
              public MyThread(ScriptEngine engine, CompiledScript script) {
                  this.engine = engine;
                  this.script = script;
              }
      
              public void run() {
                  TestClazz testClazz = new TestClazz(new Random().nextInt());
                  Map<String, Object> params = new HashMap<String, Object>();
                  params.put("clazz", testClazz);
                  Bindings bindings = engine.createBindings();
                  bindings.putAll(params);
                  try {
                      System.out.println(script.eval(bindings));
                  } catch (ScriptException e) {
                      e.printStackTrace();
                  }
              }
          }
      

      And I make 10 thread every 50 ms.

      TodoJava.java
          public static void main(String[] args) throws Exception {
              ScriptEngineManager m = new ScriptEngineManager();
              ScriptEngine engine = m.getEngineByName("groovy");
              CompiledScript script = ((Compilable) engine).compile(new FileReader( "test-script.groovy" ));
      
              InputStreamReader is_reader = new InputStreamReader(System.in);
              String str = new BufferedReader(is_reader).readLine();
      
              while (true) {
                  for (int i = 0; i < 10; ++i) {
                      MyThread myThread = new MyThread(engine, script);
                      myThread.start();
                  }
                  Thread.sleep(50);
              }
          }
      
      

      After a while, I see 90 java_util_List$iterator and 140 java_util_Random$nextInt in dump file! There are also 4 java_util_List$get but they are java_util_List$get$89 to java_util_List$get$92. I guess the rest are collected somehow. Although the "method classes" won't increase anymore, but those java_util_List$iterator and java_util_Random$nextInt will never be collected!

      I guess the number of those "method classes" is related to the number and frequency of threads excuting the groovy script. Then I try to use groovy class loader to compile a class to excute. I write a TestGroovy.groovy:

      TestGroovy.groovy
      import cn.ming.youxi.Parent
      import cn.ming.youxi.TestClazz
      
      class TestGroovy implements Parent {
          @Override
          void run() {
              Random random = new Random();
              List list = Arrays.asList(
      
                      random.nextInt(),
                      random.nextInt(),
                      random.nextInt(),
                      random.nextInt(),
              )
              list.get(2);
              println(random.nextInt());
          }
      }
      
      

      And the code to compile and call it:

      TodoJava.java
          public static void main(String[] args) throws Exception {
              GroovyClassLoader loader = new GroovyClassLoader();
              Class clazz = loader.parseClass(new File("TestGroovy.groovy"));
              Parent parent = (Parent) clazz.newInstance();
      
              InputStreamReader is_reader = new InputStreamReader(System.in);
              String str = new BufferedReader(is_reader).readLine();
      
              while (true){
                  for (int i = 0; i<10; ++i) {
                      MyGroovyClassThread myGroovyClassThread = new MyGroovyClassThread(parent);
                      myGroovyClassThread.run();
                  }
                  Thread.sleep(50);
              }
      
          }
      
          static class MyGroovyClassThread extends Thread {
              Parent parent;
              boolean interupt = false;
              public MyGroovyClassThread(Parent parent) {
                  this.parent = parent;
              }
              public MyGroovyClassThread(Parent parent, boolean interupt) {
                  this.parent = parent;
                  this.interupt = interupt;
              }
      
              public void run() {
                  parent.run();
              }
          }
      

      There are java_util_Random$nextInt and java_util_List$get all the same. However the number of each "method class" is only one.

      I think this is a critical bug in groovy, cause in fact I didn't compile the code many times but the script still creates large number of "method classes". If I use groovy script in my server, in order to prevent my website oom, I have to adjust the size of permgen!

      Here is my all java test code:

      TodoJava.java
      package cn.ming.youxi;
      
      import groovy.lang.GroovyClassLoader;
      import groovy.lang.Script;
      
      import javax.script.*;
      import java.io.*;
      import java.net.URISyntaxException;
      import java.util.HashMap;
      import java.util.Map;
      import java.util.Random;
      
      public class TodoJava {
      
          public static void main2(String[] args) throws ScriptException, IOException, InterruptedException {
              ScriptEngineManager m = new ScriptEngineManager();
              ScriptEngine engine = m.getEngineByName("groovy");
              CompiledScript script = ((Compilable) engine).compile(new FileReader("/Users/liming_liu/Documents/workspace/groovy-demo/test-script.groovy"));
      
              InputStreamReader is_reader = new InputStreamReader(System.in);
              String str = new BufferedReader(is_reader).readLine();
      
              while (true) {
                  for (int i = 0; i < 10; ++i) {
                      MyThread myThread = new MyThread(engine, script);
                      myThread.start();
                  }
                  Thread.sleep(50);
              }
          }
      
          static class MyThread extends Thread {
              ScriptEngine engine;
              CompiledScript script;
              public MyThread(ScriptEngine engine, CompiledScript script) {
                  this.engine = engine;
                  this.script = script;
              }
      
              public void run() {
                  TestClazz testClazz = new TestClazz(new Random().nextInt());
                  Map<String, Object> params = new HashMap<String, Object>();
                  params.put("clazz", testClazz);
                  Bindings bindings = engine.createBindings();
                  bindings.putAll(params);
                  try {
                      System.out.println(script.eval(bindings));
                  } catch (ScriptException e) {
                      e.printStackTrace();
                  }
              }
          }
      
          public static void main1(String[] args) throws ScriptException, IOException, IllegalAccessException, InstantiationException, InterruptedException {
              GroovyClassLoader loader = new GroovyClassLoader();
              Class clazz = loader.parseClass(new File("/Users/liming_liu/Documents/workspace/groovy-demo/test-script.groovy"));
              Random random = new Random();
              Script groovyClass = (Script) clazz.newInstance();
      
              InputStreamReader is_reader = new InputStreamReader(System.in);
              String str = new BufferedReader(is_reader).readLine();
      
              while (true) {
                  for (int i = 0; i < 10; ++i) {
                      MyScriptClassThread myThread = new MyScriptClassThread(groovyClass);
                      myThread.start();
                  }
                  Thread.sleep(50);
              }
          }
      
          static class MyScriptClassThread extends Thread {
              Script groovyClass;
              boolean interupt = false;
              public MyScriptClassThread(Script groovyClass) {
                  this.groovyClass = groovyClass;
              }
              public MyScriptClassThread(Script groovyClass, boolean interupt) {
                  this.groovyClass = groovyClass;
                  this.interupt = interupt;
              }
      
              public void run() {
                  TestClazz testClazz = new TestClazz(new Random().nextInt());
                  Map<String, Object> params = new HashMap<String, Object>();
                  params.put("clazz", testClazz);
                  groovyClass.setProperty("clazz", testClazz);
                  System.out.println(groovyClass.run());
              }
          }
      
      
          public static void main(String[] args) throws URISyntaxException, IOException, IllegalAccessException, InstantiationException, InterruptedException {
              GroovyClassLoader loader = new GroovyClassLoader();
              Class clazz = loader.parseClass(new File("/Users/liming_liu/Documents/workspace/groovy-demo/TestGroovy.groovy"));
              Parent parent = (Parent) clazz.newInstance();
      
              InputStreamReader is_reader = new InputStreamReader(System.in);
              String str = new BufferedReader(is_reader).readLine();
      
      
              while (true){
                  for (int i = 0; i<10; ++i) {
                      MyGroovyClassThread myGroovyClassThread = new MyGroovyClassThread(parent);
                      myGroovyClassThread.run();
                  }
                  Thread.sleep(50);
              }
      
          }
      
          static class MyGroovyClassThread extends Thread {
              Parent parent;
              boolean interupt = false;
              public MyGroovyClassThread(Parent parent) {
                  this.parent = parent;
              }
              public MyGroovyClassThread(Parent parent, boolean interupt) {
                  this.parent = parent;
                  this.interupt = interupt;
              }
      
              public void run() {
                  parent.run();
              }
          }
      }
      
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jwagenleitner John Wagenleitner
                Reporter:
                daqingllm Troy Liu
              • Votes:
                1 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: