Uploaded image for project: 'Groovy'
  1. Groovy
  2. GROOVY-7109

Severe Memory Leak Happens when Script Executes

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 2.3.6
    • 2.4.8
    • groovy-runtime
    • None

    Description

      Recently I found my website creates so many classes in permGen. In dump file I can see so many strange classes such as java_util_List$size, java_util_List$subList.

      I don't know what are these classes, so I just call them "method classes". The serious thing is that, each "method class" has many "anonymous classes", such as java_util_List$size$0, java_util_List$size$1, java_util_List$size$2 ... There are over 1,000 java_util_List$size, and over 60,000 class totally in permGen.

      Then I test the groovy script engine and find that when script is excuting, these strange method is created.
      Here is my test code.

      TodoJava.java
              ScriptEngineManager m = new ScriptEngineManager();
        ScriptEngine engine = m.getEngineByName("groovy");
        CompiledScript script = ((Compilable) engine).compile(new FileReader( "test-script.groovy" ));

      The test-script.groovy file:

      test-script.groovy
      import cn.ming.youxi.TestClazz

Random random = new Random();
for (i in 1 .. 10) {
    random.nextInt(i);
}
TestClazz testClazz = clazz;
List list = Arrays.asList(
        testClazz.att,
        testClazz.att,
        testClazz.att,
        testClazz.att,
        testClazz.att,
        testClazz.att
);

return list.get(2);

      TestClazz is a simple class with an Integer attribute. When I call this method, the "method classes" is created.

      TodoJava.java
              public void run(CompiledScript script) {
            TestClazz testClazz = new TestClazz(new Random().nextInt());
            Map<String, Object> params = new HashMap<String, Object>();
            params.put("clazz", testClazz);
            Bindings bindings = engine.createBindings();
            bindings.putAll(params);
            try {
                System.out.println(script.eval(bindings));
            } catch (ScriptException e) {
                e.printStackTrace();
            }
        }

      But there is only one java_util_List$get and java_util_Random$nextInt no matter how many times I call it. Then I try to execute the script in muti-thread. Here's the code:

      TodoJava.java
          static class MyThread extends Thread {
        ScriptEngine engine;
        CompiledScript script;
        public MyThread(ScriptEngine engine, CompiledScript script) {
            this.engine = engine;
            this.script = script;
        }

        public void run() {
            TestClazz testClazz = new TestClazz(new Random().nextInt());
            Map<String, Object> params = new HashMap<String, Object>();
            params.put("clazz", testClazz);
            Bindings bindings = engine.createBindings();
            bindings.putAll(params);
            try {
                System.out.println(script.eval(bindings));
            } catch (ScriptException e) {
                e.printStackTrace();
            }
        }
    }

      And I make 10 thread every 50 ms.

      TodoJava.java
          public static void main(String[] args) throws Exception {
        ScriptEngineManager m = new ScriptEngineManager();
        ScriptEngine engine = m.getEngineByName("groovy");
        CompiledScript script = ((Compilable) engine).compile(new FileReader( "test-script.groovy" ));

        InputStreamReader is_reader = new InputStreamReader(System.in);
        String str = new BufferedReader(is_reader).readLine();

        while (true) {
            for (int i = 0; i < 10; ++i) {
                MyThread myThread = new MyThread(engine, script);
                myThread.start();
            }
            Thread.sleep(50);
        }
    }

      After a while, I see 90 java_util_List$iterator and 140 java_util_Random$nextInt in dump file! There are also 4 java_util_List$get but they are java_util_List$get$89 to java_util_List$get$92. I guess the rest are collected somehow. Although the "method classes" won't increase anymore, but those java_util_List$iterator and java_util_Random$nextInt will never be collected!

      I guess the number of those "method classes" is related to the number and frequency of threads excuting the groovy script. Then I try to use groovy class loader to compile a class to excute. I write a TestGroovy.groovy:

      TestGroovy.groovy
      import cn.ming.youxi.Parent
import cn.ming.youxi.TestClazz

class TestGroovy implements Parent {
    @Override
    void run() {
        Random random = new Random();
        List list = Arrays.asList(

                random.nextInt(),
                random.nextInt(),
                random.nextInt(),
                random.nextInt(),
        )
        list.get(2);
        println(random.nextInt());
    }
}

      And the code to compile and call it:

      TodoJava.java
          public static void main(String[] args) throws Exception {
        GroovyClassLoader loader = new GroovyClassLoader();
        Class clazz = loader.parseClass(new File("TestGroovy.groovy"));
        Parent parent = (Parent) clazz.newInstance();

        InputStreamReader is_reader = new InputStreamReader(System.in);
        String str = new BufferedReader(is_reader).readLine();

        while (true){
            for (int i = 0; i<10; ++i) {
                MyGroovyClassThread myGroovyClassThread = new MyGroovyClassThread(parent);
                myGroovyClassThread.run();
            }
            Thread.sleep(50);
        }

    }

    static class MyGroovyClassThread extends Thread {
        Parent parent;
        boolean interupt = false;
        public MyGroovyClassThread(Parent parent) {
            this.parent = parent;
        }
        public MyGroovyClassThread(Parent parent, boolean interupt) {
            this.parent = parent;
            this.interupt = interupt;
        }

        public void run() {
            parent.run();
        }
    }

      There are java_util_Random$nextInt and java_util_List$get all the same. However the number of each "method class" is only one.

      I think this is a critical bug in groovy, cause in fact I didn't compile the code many times but the script still creates large number of "method classes". If I use groovy script in my server, in order to prevent my website oom, I have to adjust the size of permgen!

      Here is my all java test code:

      TodoJava.java
      package cn.ming.youxi;

import groovy.lang.GroovyClassLoader;
import groovy.lang.Script;

import javax.script.*;
import java.io.*;
import java.net.URISyntaxException;
import java.util.HashMap;
import java.util.Map;
import java.util.Random;

public class TodoJava {

    public static void main2(String[] args) throws ScriptException, IOException, InterruptedException {
        ScriptEngineManager m = new ScriptEngineManager();
        ScriptEngine engine = m.getEngineByName("groovy");
        CompiledScript script = ((Compilable) engine).compile(new FileReader("/Users/liming_liu/Documents/workspace/groovy-demo/test-script.groovy"));

        InputStreamReader is_reader = new InputStreamReader(System.in);
        String str = new BufferedReader(is_reader).readLine();

        while (true) {
            for (int i = 0; i < 10; ++i) {
                MyThread myThread = new MyThread(engine, script);
                myThread.start();
            }
            Thread.sleep(50);
        }
    }

    static class MyThread extends Thread {
        ScriptEngine engine;
        CompiledScript script;
        public MyThread(ScriptEngine engine, CompiledScript script) {
            this.engine = engine;
            this.script = script;
        }

        public void run() {
            TestClazz testClazz = new TestClazz(new Random().nextInt());
            Map<String, Object> params = new HashMap<String, Object>();
            params.put("clazz", testClazz);
            Bindings bindings = engine.createBindings();
            bindings.putAll(params);
            try {
                System.out.println(script.eval(bindings));
            } catch (ScriptException e) {
                e.printStackTrace();
            }
        }
    }

    public static void main1(String[] args) throws ScriptException, IOException, IllegalAccessException, InstantiationException, InterruptedException {
        GroovyClassLoader loader = new GroovyClassLoader();
        Class clazz = loader.parseClass(new File("/Users/liming_liu/Documents/workspace/groovy-demo/test-script.groovy"));
        Random random = new Random();
        Script groovyClass = (Script) clazz.newInstance();

        InputStreamReader is_reader = new InputStreamReader(System.in);
        String str = new BufferedReader(is_reader).readLine();

        while (true) {
            for (int i = 0; i < 10; ++i) {
                MyScriptClassThread myThread = new MyScriptClassThread(groovyClass);
                myThread.start();
            }
            Thread.sleep(50);
        }
    }

    static class MyScriptClassThread extends Thread {
        Script groovyClass;
        boolean interupt = false;
        public MyScriptClassThread(Script groovyClass) {
            this.groovyClass = groovyClass;
        }
        public MyScriptClassThread(Script groovyClass, boolean interupt) {
            this.groovyClass = groovyClass;
            this.interupt = interupt;
        }

        public void run() {
            TestClazz testClazz = new TestClazz(new Random().nextInt());
            Map<String, Object> params = new HashMap<String, Object>();
            params.put("clazz", testClazz);
            groovyClass.setProperty("clazz", testClazz);
            System.out.println(groovyClass.run());
        }
    }


    public static void main(String[] args) throws URISyntaxException, IOException, IllegalAccessException, InstantiationException, InterruptedException {
        GroovyClassLoader loader = new GroovyClassLoader();
        Class clazz = loader.parseClass(new File("/Users/liming_liu/Documents/workspace/groovy-demo/TestGroovy.groovy"));
        Parent parent = (Parent) clazz.newInstance();

        InputStreamReader is_reader = new InputStreamReader(System.in);
        String str = new BufferedReader(is_reader).readLine();


        while (true){
            for (int i = 0; i<10; ++i) {
                MyGroovyClassThread myGroovyClassThread = new MyGroovyClassThread(parent);
                myGroovyClassThread.run();
            }
            Thread.sleep(50);
        }

    }

    static class MyGroovyClassThread extends Thread {
        Parent parent;
        boolean interupt = false;
        public MyGroovyClassThread(Parent parent) {
            this.parent = parent;
        }
        public MyGroovyClassThread(Parent parent, boolean interupt) {
            this.parent = parent;
            this.interupt = interupt;
        }

        public void run() {
            parent.run();
        }
    }
}

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. GROOVY-7683 Memory leak when using Groovy as JSR-223 scripting language.

          • Major - Major loss of function.
          • Closed

          Activity

            People

              jwagenleitner John Wagenleitner
              daqingllm Troy Liu
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: