Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1885

calcite code gen make many class in perm and it cant be gc

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      calcite code gen make many class in perm and it cant be gc, how to unload that class, i need help ! thx!
      this is log for cluster env , i find it locked in classload and Class.forname01:
      java.lang.Thread.State: BLOCKED (on object monitor)
      at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1735)

      • waiting to lock <0x0000000340001cf8> (a io.kyligence.kap.ext.CustomizedWebappClassloader)
        at io.kyligence.kap.ext.CustomizedWebappClassloader.loadClass(CustomizedWebappClassloader.java:76)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
      • locked <0x00000005f9805b18> (a org.codehaus.janino.ByteArrayClassLoader)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at Baz.bind(Unknown Source)
        at org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:331)
        at org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:294)
        at org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:553)
        at org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:544)
        at org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:193)
        at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
        at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)

      java.lang.Thread.State: BLOCKED (on object monitor)
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:348)
      at org.codehaus.janino.ClassLoaderIClassLoader.findIClass(ClassLoaderIClassLoader.java:78)
      at org.codehaus.janino.IClassLoader.loadIClass(IClassLoader.java:254)

      • locked <0x00000005e9802818> (a org.codehaus.janino.ClassLoaderIClassLoader)

      this test case can reappear memory leak ,use it with csvtest!

      BasicDataSource dataSource = (BasicDataSource) BasicDataSourceFactory.createDataSource(new Properties());

      private void checkSql(String sql, String model, Function<ResultSet, Void> fn)
      throws SQLException {
      Connection connection = null;
      Statement statement = null;
      try

      { Properties info = new Properties(); info.put("model", jsonPath(model)); dataSource.setDriverClassName(Driver.class.getName()); dataSource.setUrl("jdbc:calcite:model=" + jsonPath(model)); connection = dataSource.getConnection(); statement = connection.createStatement(); final ResultSet resultSet = statement.executeQuery( sql); fn.apply(resultSet); }

      catch (Exception e)

      { e.printStackTrace(); }

      finally

      { close(connection, statement); }

      }

      the test case:
      @Test
      public void testConcurrentQuery() throws SQLException, InterruptedException {
      final String sql = "select * \n"
      + "from long_emps ";
      final AtomicLong a = new AtomicLong(0);
      long start = System.currentTimeMillis();
      final Semaphore semaphore = new Semaphore(0);
      for (int i = 0; i < 8; i++) {

      new Thread(new Runnable() {
      int count = 0;

      @Override
      public void run() {
      while (true) {
      sql("bug", sql).checking(new Function<ResultSet, Void>() {
      public Void apply(ResultSet resultSet)

      { return null; }

      }).ok();
      semaphore.release();
      // System.out.println(Thread.currentThread().getName() + ":" + count);
      }
      }
      }).start();

      }
      semaphore.acquire(1000000);
      System.out.println(System.currentTimeMillis() - start);
      }

        Activity

        Hide
        julianhyde Julian Hyde added a comment -

        I saw you closed this as "fixed". But I don't see a fix.

        Can you please describe how you solved the problem? I want to make sure that others who run into the same problem have a solution.

        Show
        julianhyde Julian Hyde added a comment - I saw you closed this as "fixed". But I don't see a fix. Can you please describe how you solved the problem? I want to make sure that others who run into the same problem have a solution.
        Hide
        hn5092 yiming.xu added a comment - - edited

        i am retest use -Xmx100m g1 gc and use calcite csv test , i find it can be full gc, but in cluster env, we gave it 1g permsize , it still oom ,Is it because the concurrency is too high?

        Show
        hn5092 yiming.xu added a comment - - edited i am retest use -Xmx100m g1 gc and use calcite csv test , i find it can be full gc, but in cluster env, we gave it 1g permsize , it still oom ,Is it because the concurrency is too high?
        Hide
        julianhyde Julian Hyde added a comment -

        Makes sure you're closing everything that can be closed. If statements, result sets and enumerables are still open maybe their classes cannot be garbage collected.

        CompilerFactoryFactory.getDefaultCompilerFactory() doesn't look thread safe. If your concurrency is really high maybe you've ended up with more than one. Try synchronizing Calcite's calls to that method.

        Show
        julianhyde Julian Hyde added a comment - Makes sure you're closing everything that can be closed. If statements, result sets and enumerables are still open maybe their classes cannot be garbage collected. CompilerFactoryFactory.getDefaultCompilerFactory() doesn't look thread safe. If your concurrency is really high maybe you've ended up with more than one. Try synchronizing Calcite's calls to that method.
        Hide
        hn5092 yiming.xu added a comment - - edited

        Hi !Julian, I looked at the high jspack scene, a lot of threads in the calcite code generation link block in the classload.load, there are class.forname0 above, then I found calcite code generation will call CompilerFactory.newClassBodyEvaluator, this time with To the ByteArraysClassload, but after my observation found that the classload has a problem, he will first to the higher class load query the current need to load the class is the parent class load load, which led to each thread bytearrayclassload.load Go to the root classload. But we know that the code generated by the class is definitely not the father of the load, so I revised the next classload, first call their own findclass, and then can not find the next class to find, The congestion in the high concurrent scene is less, but I do not know if there is any risk in this design?

        Show
        hn5092 yiming.xu added a comment - - edited Hi !Julian, I looked at the high jspack scene, a lot of threads in the calcite code generation link block in the classload.load, there are class.forname0 above, then I found calcite code generation will call CompilerFactory.newClassBodyEvaluator, this time with To the ByteArraysClassload, but after my observation found that the classload has a problem, he will first to the higher class load query the current need to load the class is the parent class load load, which led to each thread bytearrayclassload.load Go to the root classload. But we know that the code generated by the class is definitely not the father of the load, so I revised the next classload, first call their own findclass, and then can not find the next class to find, The congestion in the high concurrent scene is less, but I do not know if there is any risk in this design?
        Hide
        julianhyde Julian Hyde added a comment -

        I don't quite understand what you are proposing. Can you create a pull request so I can review?

        Show
        julianhyde Julian Hyde added a comment - I don't quite understand what you are proposing. Can you create a pull request so I can review?
        Hide
        hn5092 yiming.xu added a comment -

        Hi !Julian,I have a new discovery, if i use dbcp connect pool , the bz class cant be gc , because CalciteSignature have it referring. i close connection still invalid, Is this a bug?

        Show
        hn5092 yiming.xu added a comment - Hi !Julian,I have a new discovery, if i use dbcp connect pool , the bz class cant be gc , because CalciteSignature have it referring. i close connection still invalid, Is this a bug?
        Hide
        julianhyde Julian Hyde added a comment -

        This is not surprising. If you use a connection pool the connections in the pool are not closed, only the wrapper that the connection pool creates on top of the connections is closed. That said, I'd hope that the generated class could be garbage collected when only the statement is closed; it should not have to wait for the connection.

        Have you tried using just a small pool (say 5 connections)?

        Show
        julianhyde Julian Hyde added a comment - This is not surprising. If you use a connection pool the connections in the pool are not closed, only the wrapper that the connection pool creates on top of the connections is closed. That said, I'd hope that the generated class could be garbage collected when only the statement is closed; it should not have to wait for the connection. Have you tried using just a small pool (say 5 connections)?

          People

          • Assignee:
            julianhyde Julian Hyde
            Reporter:
            hn5092 yiming.xu
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development