Derby
  1. Derby
  2. DERBY-4773

Derby Network service mode creates a lot of generated classes (extends BaseActivation).

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 10.5.3.0
    • Fix Version/s: None
    • Component/s: SQL
    • Environment:
      Suse Linux 11, Hive 5.0, Hadoop v.20.1
    • Urgency:
      Normal

      Description

      When I run Derby as a Network service, due to some reason it created about 1800 generated calss files (ac0a3c3082x012ax5c4fx64fcx00004650e8f2b.class, extends BaseActivation), I am unable to figure out the reason and now unable to reproduce the situation again. Has anyone faced this issue.

        Issue Links

          Activity

          Hide
          Knut Anders Hatlen added a comment -

          You could check the list of open data corruption issues and see if you find something that looks similar to what you're seeing. The list can be found here: https://issues.apache.org/jira/secure/IssueNavigator.jspa?mode=hide&requestId=12313481

          Show
          Knut Anders Hatlen added a comment - You could check the list of open data corruption issues and see if you find something that looks similar to what you're seeing. The list can be found here: https://issues.apache.org/jira/secure/IssueNavigator.jspa?mode=hide&requestId=12313481
          Hide
          Soundararajan Velu added a comment -

          It makes sense, Thanks Knut, we are facing both the issue, "ArrayIndexoutofBounds" and intermittent DB corruption. Our database is automatically generated using thirdparty libraries, This could be a potential reason,

          Is there anyother means by which derby database can be corrupted during normal operations. Since we are unable to predict or correlate the corruption reason and frequency.

          Show
          Soundararajan Velu added a comment - It makes sense, Thanks Knut, we are facing both the issue, "ArrayIndexoutofBounds" and intermittent DB corruption. Our database is automatically generated using thirdparty libraries, This could be a potential reason, Is there anyother means by which derby database can be corrupted during normal operations. Since we are unable to predict or correlate the corruption reason and frequency.
          Hide
          Knut Anders Hatlen added a comment -

          The stack trace of the ArrayIndexOutOfBoundsException is similar to the one we saw in DERBY-2584. Maybe the fix for that bug wasn't complete.

          Show
          Knut Anders Hatlen added a comment - The stack trace of the ArrayIndexOutOfBoundsException is similar to the one we saw in DERBY-2584 . Maybe the fix for that bug wasn't complete.
          Hide
          Soundararajan Velu added a comment -

          Thanks Rick, This issue occurs with the version 10.5.3.0, I quoted a wrong version initially.
          Basically a heartbeat update script constantly connects executes a simple query and disconnects.

          Show
          Soundararajan Velu added a comment - Thanks Rick, This issue occurs with the version 10.5.3.0, I quoted a wrong version initially. Basically a heartbeat update script constantly connects executes a simple query and disconnects.
          Hide
          Rick Hillegas added a comment -

          Thanks. It looks to me as though we are tripping across an internal error while processing a client's call to DatabaseMetaData.getIndexInfo() on a table called DBS. I have tried to reproduce the trace by calling getIndexInfo( null, null "T", true, true ), but that call succeeds for me. Do you have a little script/program which triggers this stack trace and which you could attach? Thanks.

          Show
          Rick Hillegas added a comment - Thanks. It looks to me as though we are tripping across an internal error while processing a client's call to DatabaseMetaData.getIndexInfo() on a table called DBS. I have tried to reproduce the trace by calling getIndexInfo( null, null "T", true, true ), but that call succeeds for me. Do you have a little script/program which triggers this stack trace and which you could attach? Thanks.
          Hide
          Soundararajan Velu added a comment -

          Rick this is the stack trace for the issue, It would be great if you can suggest...

          2010-08-12 00:11:28.913 GMT : Apache Derby Network Server - 10.5.3.0 - (802917) started and ready to accept connections on port 1527
          ----------------------------------------------------------------
          2010-08-12 00:11:33.926 GMT:
           Booting Derby version The Apache Software Foundation - Apache Derby - 10.5.3.0 - (802917): instance a816c00e-012a-63a3-1ecc-00004650e8f2
          on database directory /ngcdn/md/derby/bin/metastore_db  
          
          Database Class Loader started - derby.database.classpath=''
          2010-08-12 00:11:42.010 GMT Thread[DRDAConnThread_3,5,main] (XID = 1331860), (SESSIONID = 1), (DATABASE = metastore_db), (DRDAID = C0A86101.C9BA-4413244878274481857{1}), Cleanup action starting
          2010-08-12 00:11:42.010 GMT Thread[DRDAConnThread_3,5,main] (XID = 1331860), (SESSIONID = 1), (DATABASE = metastore_db), (DRDAID = C0A86101.C9BA-4413244878274481857{1}), Failed Statement is: CALL SYSIBM.SQLSTATISTICS(?,?,?,?,?,?) with 6 parameters begin parameter #1: NULL :end parameter begin parameter #2: NULL :end parameter begin parameter #3: DBS :end parameter begin parameter #4: 1 :end parameter begin parameter #5: 1 :end parameter begin parameter #6: DATATYPE='JDBC';DYNAMIC=0;REPORTPUBLICPRIVILEGES=1;CURSORHOLD=1 :end parameter 
          java.lang.ArrayIndexOutOfBoundsException: 1
          	at org.apache.derby.impl.sql.GenericParameterValueSet.initialize(Unknown Source)
          	at org.apache.derby.impl.sql.execute.BaseActivation.setupActivation(Unknown Source)
          	at org.apache.derby.impl.sql.GenericActivationHolder.<init>(Unknown Source)
          	at org.apache.derby.impl.sql.GenericPreparedStatement.getActivation(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedPreparedStatement20.<init>(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedPreparedStatement30.<init>(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedPreparedStatement40.<init>(Unknown Source)
          	at org.apache.derby.jdbc.Driver40.newEmbedPreparedStatement(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedConnection.prepareMetaDataStatement(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.prepareSPS(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.getPreparedQueryUsingSystemTables(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.getPreparedQuery(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.getPreparedQuery(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.doGetIndexInfo(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.getIndexInfo(Unknown Source)
          	at org.apache.derby.catalog.SystemProcedures.SQLSTATISTICS(Unknown Source)
          	at org.apache.derby.exe.acf81e0010x012ax63a3x1eccx00004650e8f21b.g0(Unknown Source)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
          	at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
          	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
          	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedCallableStatement.executeStatement(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
          Cleanup action completed
          New exception raised during cleanup null
          java.lang.NullPointerException
          	at org.apache.derby.impl.sql.compile.CompilerContextImpl.initRequiredPriv(Unknown Source)
          	at org.apache.derby.impl.sql.compile.CompilerContextImpl.resetContext(Unknown Source)
          	at org.apache.derby.impl.sql.compile.CompilerContextImpl.cleanupOnError(Unknown Source)
          	at org.apache.derby.iapi.services.context.ContextManager.cleanupOnError(Unknown Source)
          	at org.apache.derby.impl.jdbc.TransactionResourceImpl.cleanupOnError(Unknown Source)
          	at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
          	at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedCallableStatement.executeStatement(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
          2010-08-12 00:11:42.850 GMT Thread[DRDAConnThread_3,5,main] Equally severe exception raised during cleanup (ignored) null
          java.lang.NullPointerException
          	at org.apache.derby.impl.sql.compile.CompilerContextImpl.initRequiredPriv(Unknown Source)
          	at org.apache.derby.impl.sql.compile.CompilerContextImpl.resetContext(Unknown Source)
          	at org.apache.derby.impl.sql.compile.CompilerContextImpl.cleanupOnError(Unknown Source)
          	at org.apache.derby.iapi.services.context.ContextManager.cleanupOnError(Unknown Source)
          	at org.apache.derby.impl.jdbc.TransactionResourceImpl.cleanupOnError(Unknown Source)
          	at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
          	at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedCallableStatement.executeStatement(Unknown Source)
          	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
          	at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
          Cleanup action completed
          
          Show
          Soundararajan Velu added a comment - Rick this is the stack trace for the issue, It would be great if you can suggest... 2010-08-12 00:11:28.913 GMT : Apache Derby Network Server - 10.5.3.0 - (802917) started and ready to accept connections on port 1527 ---------------------------------------------------------------- 2010-08-12 00:11:33.926 GMT: Booting Derby version The Apache Software Foundation - Apache Derby - 10.5.3.0 - (802917): instance a816c00e-012a-63a3-1ecc-00004650e8f2 on database directory /ngcdn/md/derby/bin/metastore_db Database Class Loader started - derby.database.classpath='' 2010-08-12 00:11:42.010 GMT Thread[DRDAConnThread_3,5,main] (XID = 1331860), (SESSIONID = 1), (DATABASE = metastore_db), (DRDAID = C0A86101.C9BA-4413244878274481857{1}), Cleanup action starting 2010-08-12 00:11:42.010 GMT Thread[DRDAConnThread_3,5,main] (XID = 1331860), (SESSIONID = 1), (DATABASE = metastore_db), (DRDAID = C0A86101.C9BA-4413244878274481857{1}), Failed Statement is: CALL SYSIBM.SQLSTATISTICS(?,?,?,?,?,?) with 6 parameters begin parameter #1: NULL :end parameter begin parameter #2: NULL :end parameter begin parameter #3: DBS :end parameter begin parameter #4: 1 :end parameter begin parameter #5: 1 :end parameter begin parameter #6: DATATYPE='JDBC';DYNAMIC=0;REPORTPUBLICPRIVILEGES=1;CURSORHOLD=1 :end parameter java.lang.ArrayIndexOutOfBoundsException: 1 at org.apache.derby.impl.sql.GenericParameterValueSet.initialize(Unknown Source) at org.apache.derby.impl.sql.execute.BaseActivation.setupActivation(Unknown Source) at org.apache.derby.impl.sql.GenericActivationHolder.<init>(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.getActivation(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement20.<init>(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement30.<init>(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement40.<init>(Unknown Source) at org.apache.derby.jdbc.Driver40.newEmbedPreparedStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.prepareMetaDataStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.prepareSPS(Unknown Source) at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.getPreparedQueryUsingSystemTables(Unknown Source) at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.getPreparedQuery(Unknown Source) at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.getPreparedQuery(Unknown Source) at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.doGetIndexInfo(Unknown Source) at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.getIndexInfo(Unknown Source) at org.apache.derby.catalog.SystemProcedures.SQLSTATISTICS(Unknown Source) at org.apache.derby.exe.acf81e0010x012ax63a3x1eccx00004650e8f21b.g0(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source) at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedCallableStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source) Cleanup action completed New exception raised during cleanup null java.lang.NullPointerException at org.apache.derby.impl.sql.compile.CompilerContextImpl.initRequiredPriv(Unknown Source) at org.apache.derby.impl.sql.compile.CompilerContextImpl.resetContext(Unknown Source) at org.apache.derby.impl.sql.compile.CompilerContextImpl.cleanupOnError(Unknown Source) at org.apache.derby.iapi.services.context.ContextManager.cleanupOnError(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.cleanupOnError(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedCallableStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source) 2010-08-12 00:11:42.850 GMT Thread[DRDAConnThread_3,5,main] Equally severe exception raised during cleanup (ignored) null java.lang.NullPointerException at org.apache.derby.impl.sql.compile.CompilerContextImpl.initRequiredPriv(Unknown Source) at org.apache.derby.impl.sql.compile.CompilerContextImpl.resetContext(Unknown Source) at org.apache.derby.impl.sql.compile.CompilerContextImpl.cleanupOnError(Unknown Source) at org.apache.derby.iapi.services.context.ContextManager.cleanupOnError(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.cleanupOnError(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedCallableStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source) Cleanup action completed
          Hide
          Soundararajan Velu added a comment -

          Thanks a lot Rick, this really helps.. I have checked the start-up scripts and they don't carry this flag...

          The other possibility that I suspect is when Hive is performing updates on its metastore (Derby) and Derby goes down abruptly and is brought back online. at this point of time (I guess in the clean up methods), derby spits out a lot of ArrayIndexOutOfBounds exception at the initialization methods followed by NPE at the exception blocks of the execute exception handlers. which causes the leak of the generated classes.

          I am unable to upload the stack trace (due to company policy).

          I will reproduce this problem at home and upload the trace and exact scenario.

          Show
          Soundararajan Velu added a comment - Thanks a lot Rick, this really helps.. I have checked the start-up scripts and they don't carry this flag... The other possibility that I suspect is when Hive is performing updates on its metastore (Derby) and Derby goes down abruptly and is brought back online. at this point of time (I guess in the clean up methods), derby spits out a lot of ArrayIndexOutOfBounds exception at the initialization methods followed by NPE at the exception blocks of the execute exception handlers. which causes the leak of the generated classes. I am unable to upload the stack trace (due to company policy). I will reproduce this problem at home and upload the trace and exact scenario.
          Hide
          Rick Hillegas added a comment -

          Each generated class file represents the query plan for an SQL statement processed by Derby's SQL interpreter. Those generated classes extend BaseActivation and have names which look like the class names you are seeing. Normally, the generated classes are internal to Derby. However, you can cause Derby to dump the class files to disk by setting the following flag when you start the VM:

          -Dderby.debug.true=DumpClassFile

          I don't know any other way to force Derby to dump those class files to disk. If you figure out how you coaxed Derby into dumping those class files, that will help us debug this problem. Thanks.

          Show
          Rick Hillegas added a comment - Each generated class file represents the query plan for an SQL statement processed by Derby's SQL interpreter. Those generated classes extend BaseActivation and have names which look like the class names you are seeing. Normally, the generated classes are internal to Derby. However, you can cause Derby to dump the class files to disk by setting the following flag when you start the VM: -Dderby.debug.true=DumpClassFile I don't know any other way to force Derby to dump those class files to disk. If you figure out how you coaxed Derby into dumping those class files, that will help us debug this problem. Thanks.

            People

            • Assignee:
              Unassigned
              Reporter:
              Soundararajan Velu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Development