Derby
  1. Derby
  2. DERBY-5221

Database cannot be started after shutdown due to ClassNotFoundException: ERROR XBM0U: No class was registered for identifier 33089

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: 10.5.3.0
    • Fix Version/s: None
    • Component/s: Store
    • Environment:
    • Urgency:
      Normal
    • Bug behavior facts:
      Crash, Data corruption, Seen in production

      Description

      Database cannot be started after shutdown due to ClassNotFoundException: ERROR XBM0U: No class was registered for identifier 33089

      2011-05-09 11:18:45.804 GMT:
      Shutting down instance fb160050-012f-d47a-b11d-000012951948
      ----------------------------------------------------------------
      2011-05-09 11:18:45.806 GMT Thread[http-8091-1,5,main] Cleanup action starting
      java.sql.SQLException: Failed to start database '/matrix/hudson/.artifactory/data/db', see the next exception for details.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
      at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
      at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
      at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
      at java.sql.DriverManager.getConnection(DriverManager.java:582)
      at java.sql.DriverManager.getConnection(DriverManager.java:185)
      at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionFactory.getConnection(ConnectionFactory.java:92)
      at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.setupConnection(ConnectionRecoveryManager.java:334)
      at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.reestablishConnection(ConnectionRecoveryManager.java:409)
      at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.getConnection(ConnectionRecoveryManager.java:159)
      at org.apache.jackrabbit.core.persistence.bundle.BundleDbPersistenceManager.prepareSchemaObjectPrefix(BundleDbPersistenceManager.java:923
      )
      at org.apache.jackrabbit.core.persistence.bundle.BundleDbPersistenceManager.init(BundleDbPersistenceManager.java:562)
      at org.apache.jackrabbit.core.persistence.bundle.DerbyPersistenceManager.init(DerbyPersistenceManager.java:254)
      at org.apache.jackrabbit.core.RepositoryImpl.createPersistenceManager(RepositoryImpl.java:1288)
      at org.apache.jackrabbit.core.RepositoryImpl.access$800(RepositoryImpl.java:109)
      at org.apache.jackrabbit.core.RepositoryImpl$WorkspaceInfo.doInitialize(RepositoryImpl.java:1851)
      at org.apache.jackrabbit.core.RepositoryImpl$WorkspaceInfo.initialize(RepositoryImpl.java:1834)
      at org.apache.jackrabbit.core.RepositoryImpl.initStartupWorkspaces(RepositoryImpl.java:483)
      at org.apache.jackrabbit.core.RepositoryImpl.<init>(RepositoryImpl.java:324)
      at org.apache.jackrabbit.core.RepositoryImpl.create(RepositoryImpl.java:621)
      at org.artifactory.jcr.JcrRepoInitStrategy.createJcrRepository(JcrRepoInitStrategy.java:92)
      at org.artifactory.jcr.JcrServiceImpl.initJcrRepository(JcrServiceImpl.java:648)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanP
      ostProcessor.java:297)
      at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnn
      otationBeanPostProcessor.java:250)
      at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotat
      ionBeanPostProcessor.java:144)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutow
      ireCapableBeanFactory.java:350)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:13
      31)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
      at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
      at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:671)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:610)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBean
      PostProcessor.java:412)
      at org.springframework.beans.factory.annotation.InjectionMetadata.injectFields(InjectionMetadata.java:105)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessAfterInstantiation(AutowiredAnnotationBea
      nPostProcessor.java:240)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:959)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:472)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
      at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
      at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:671)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:610)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBean
      PostProcessor.java:412)
      at org.springframework.beans.factory.annotation.InjectionMetadata.injectFields(InjectionMetadata.java:105)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessAfterInstantiation(AutowiredAnnotationBea
      nPostProcessor.java:240)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:959)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:472)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
      at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
      at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:671)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:610)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBean
      PostProcessor.java:412)
      at org.springframework.beans.factory.annotation.InjectionMetadata.injectFields(InjectionMetadata.java:105)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessAfterInstantiation(AutowiredAnnotationBea
      nPostProcessor.java:240)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:959)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:472)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
      at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
      at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:671)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:610)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBean
      PostProcessor.java:412)
      at org.springframework.beans.factory.annotation.InjectionMetadata.injectFields(InjectionMetadata.java:105)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessAfterInstantiation(AutowiredAnnotationBea
      nPostProcessor.java:240)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:959)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:472)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
      at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
      at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:429)
      at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:728)
      at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:380)
      at org.artifactory.spring.ArtifactoryApplicationContext.refresh(ArtifactoryApplicationContext.java:188)
      at org.artifactory.spring.ArtifactoryApplicationContext.<init>(ArtifactoryApplicationContext.java:95)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
      at org.artifactory.webapp.servlet.ArtifactoryContextConfigListener.configure(ArtifactoryContextConfigListener.java:108)
      at org.artifactory.webapp.servlet.ArtifactoryContextConfigListener.contextInitialized(ArtifactoryContextConfigListener.java:57)
      at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3934)
      at org.apache.catalina.core.StandardContext.start(StandardContext.java:4429)
      at org.apache.catalina.manager.ManagerServlet.start(ManagerServlet.java:1249)
      at org.apache.catalina.manager.HTMLManagerServlet.start(HTMLManagerServlet.java:612)
      at org.apache.catalina.manager.HTMLManagerServlet.doGet(HTMLManagerServlet.java:136)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
      at org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:421)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
      at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
      at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
      at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
      at java.lang.Thread.run(Thread.java:619)
      Caused by: java.sql.SQLException: Failed to start database '/matrix/hudson/.artifactory/data/db', see the next exception for details.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      ... 137 more
      Caused by: java.sql.SQLException: Java exception: 'ERROR XBM0U: No class was registered for identifier 33089.: java.lang.ClassNotFoundException'.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
      ... 134 more
      Caused by: java.lang.ClassNotFoundException: ERROR XBM0U: No class was registered for identifier 33089.
      at org.apache.derby.iapi.services.io.FormatIdInputStream.readObject(Unknown Source)
      at org.apache.derby.iapi.services.io.ArrayInputStream.readObject(Unknown Source)
      at org.apache.derby.impl.store.raw.log.Scan.getNextRecordForward(Unknown Source)
      at org.apache.derby.impl.store.raw.log.Scan.getNextRecord(Unknown Source)
      at org.apache.derby.impl.store.raw.log.FileLogger.readLogRecord(Unknown Source)
      at org.apache.derby.impl.store.raw.log.LogToFile.findCheckpoint(Unknown Source)
      at org.apache.derby.impl.store.raw.log.LogToFile.recover(Unknown Source)
      at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
      at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
      at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
      at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
      ... 134 more
      ============= begin nested exception, level (1) ===========
      java.sql.SQLException: Java exception: 'ERROR XBM0U: No class was registered for identifier 33089.: java.lang.ClassNotFoundException'.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
      at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
      at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
      at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
      at java.sql.DriverManager.getConnection(DriverManager.java:582)
      at java.sql.DriverManager.getConnection(DriverManager.java:185)
      at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionFactory.getConnection(ConnectionFactory.java:92)
      at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.setupConnection(ConnectionRecoveryManager.java:334)
      at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.reestablishConnection(ConnectionRecoveryManager.java:409)
      at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.getConnection(ConnectionRecoveryManager.java:159)
      at org.apache.jackrabbit.core.persistence.bundle.BundleDbPersistenceManager.prepareSchemaObjectPrefix(BundleDbPersistenceManager.java:923
      )
      at org.apache.jackrabbit.core.persistence.bundle.BundleDbPersistenceManager.init(BundleDbPersistenceManager.java:562)
      at org.apache.jackrabbit.core.persistence.bundle.DerbyPersistenceManager.init(DerbyPersistenceManager.java:254)
      at org.apache.jackrabbit.core.RepositoryImpl.createPersistenceManager(RepositoryImpl.java:1288)
      at org.apache.jackrabbit.core.RepositoryImpl.access$800(RepositoryImpl.java:109)
      at org.apache.jackrabbit.core.RepositoryImpl$WorkspaceInfo.doInitialize(RepositoryImpl.java:1851)
      at org.apache.jackrabbit.core.RepositoryImpl$WorkspaceInfo.initialize(RepositoryImpl.java:1834)
      at org.apache.jackrabbit.core.RepositoryImpl.initStartupWorkspaces(RepositoryImpl.java:483)
      at org.apache.jackrabbit.core.RepositoryImpl.<init>(RepositoryImpl.java:324)
      at org.apache.jackrabbit.core.RepositoryImpl.create(RepositoryImpl.java:621)
      at org.artifactory.jcr.JcrRepoInitStrategy.createJcrRepository(JcrRepoInitStrategy.java:92)
      at org.artifactory.jcr.JcrServiceImpl.initJcrRepository(JcrServiceImpl.java:648)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanP
      ostProcessor.java:297)
      at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnn
      otationBeanPostProcessor.java:250)
      at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotat
      ionBeanPostProcessor.java:144)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutow
      ireCapableBeanFactory.java:350)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:13
      31)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
      at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
      at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:671)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:610)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBean
      PostProcessor.java:412)
      at org.springframework.beans.factory.annotation.InjectionMetadata.injectFields(InjectionMetadata.java:105)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessAfterInstantiation(AutowiredAnnotationBea
      nPostProcessor.java:240)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:959)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:472)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
      at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
      at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:671)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:610)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBean
      PostProcessor.java:412)
      at org.springframework.beans.factory.annotation.InjectionMetadata.injectFields(InjectionMetadata.java:105)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessAfterInstantiation(AutowiredAnnotationBea
      nPostProcessor.java:240)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:959)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:472)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
      at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
      at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:671)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:610)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBean
      PostProcessor.java:412)
      at org.springframework.beans.factory.annotation.InjectionMetadata.injectFields(InjectionMetadata.java:105)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessAfterInstantiation(AutowiredAnnotationBea
      nPostProcessor.java:240)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:959)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:472)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
      at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
      at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:671)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:610)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBean
      PostProcessor.java:412)
      at org.springframework.beans.factory.annotation.InjectionMetadata.injectFields(InjectionMetadata.java:105)
      at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessAfterInstantiation(AutowiredAnnotationBea
      nPostProcessor.java:240)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:959)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:472)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
      at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
      at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
      at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:429)
      at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:728)
      at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:380)
      at org.artifactory.spring.ArtifactoryApplicationContext.refresh(ArtifactoryApplicationContext.java:188)
      at org.artifactory.spring.ArtifactoryApplicationContext.<init>(ArtifactoryApplicationContext.java:95)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
      at org.artifactory.webapp.servlet.ArtifactoryContextConfigListener.configure(ArtifactoryContextConfigListener.java:108)
      at org.artifactory.webapp.servlet.ArtifactoryContextConfigListener.contextInitialized(ArtifactoryContextConfigListener.java:57)
      at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3934)
      at org.apache.catalina.core.StandardContext.start(StandardContext.java:4429)
      at org.apache.catalina.manager.ManagerServlet.start(ManagerServlet.java:1249)
      at org.apache.catalina.manager.HTMLManagerServlet.start(HTMLManagerServlet.java:612)
      at org.apache.catalina.manager.HTMLManagerServlet.doGet(HTMLManagerServlet.java:136)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
      at org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:421)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
      at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
      at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
      at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
      at java.lang.Thread.run(Thread.java:619)
      Caused by: java.sql.SQLException: Java exception: 'ERROR XBM0U: No class was registered for identifier 33089.: java.lang.ClassNotFoundException'.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      ... 137 more
      Caused by: java.lang.ClassNotFoundException: ERROR XBM0U: No class was registered for identifier 33089.
      at org.apache.derby.iapi.services.io.FormatIdInputStream.readObject(Unknown Source)
      at org.apache.derby.iapi.services.io.ArrayInputStream.readObject(Unknown Source)
      at org.apache.derby.impl.store.raw.log.Scan.getNextRecordForward(Unknown Source)
      at org.apache.derby.impl.store.raw.log.Scan.getNextRecord(Unknown Source)
      at org.apache.derby.impl.store.raw.log.FileLogger.readLogRecord(Unknown Source)
      at org.apache.derby.impl.store.raw.log.LogToFile.findCheckpoint(Unknown Source)
      at org.apache.derby.impl.store.raw.log.LogToFile.recover(Unknown Source)
      at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
      at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
      at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
      at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)

      1. storeLog.tar.gz
        401 kB
        kw
      2. derbyLog.tar.gz
        19 kB
        kw
      3. derby.log.gz
        53 kB
        kw

        Issue Links

          Activity

          Hide
          SCHAEFER B.S. added a comment -

          Dear Mike,

          thanks for your mail, but let me point out the tedious thing:

          In our case the error occured late after startup on inserting records.
          A call to CALL SYSCS_UTIL.SYSCS_COMPRESS_TABLE(?, ?, ?) solved the problem
          In the past we had other erros like "The conglomerate does not exist" due to virus scans or other spooky/magic things.

          To put in a nutshell: Derby can do a "self healing" itself!!!

          I paste below the code we add as "bootstrap" after database startup to make sure, everything is allright!
          Tedious, isn't? Furthermore: We insert dummy records into all tables and deleting them immediately to find the errors like XBM0U.

          My suggestion is: Put that "self healing" code as OPTION into

          • database startup code (bootstrap)
          • or in SQLException when it occurs to try a self repair

          If you are not convinced or not agree, I don't mind -> we implemented that code for robustness

          Regards Andreas

          -------------------------------------------

          private void checkAndRepairTable(String tablename) throws SQLException {
          boolean isok = checkTable(tablename);
          if (!isok) {
          _log.log(
          Level.INFO,
          _logprefix + "Trying to repair table " + tablename + " ...");
          isok = compressTable(tablename);
          if (!isok) {
          // Rename table, delete indexes and create new
          Statement s = null;
          try {
          String renamed_table = tablename + "_DAMAGED";
          boolean docopy = true;
          try

          { renameTable(tablename, renamed_table); }

          catch (SQLException ex)

          { //If data can't be read anymore, droptable! _log.log( Level.INFO, _logprefix + "Repair table " + tablename + " failed, table can't be renamed: " + Utils.getExceptionMessage(ex)); dropTable(tablename); }

          _dbconnection_init.commit();
          // Drop indexes
          dropTableIndexes(renamed_table);
          _dbconnection_init.commit();
          s = _dbconnection_init.createStatement();

          // Drop org table and create it again
          dropTable(tablename);
          createTable(s, tablename);

          if (docopy) {
          try

          { copyTable(renamed_table, tablename); dropTable(renamed_table); }

          catch (SQLException ex)

          { _log.log( Level.WARNING, _logprefix + "Failed to repair table " + tablename + " : Could not copy data, error was : " + Utils.getExceptionMessage(ex, true)); }

          }

          } catch (Exception ex)

          { _log.log( Level.WARNING, _logprefix + "Failed to repair table " + tablename + " : " + Utils.getExceptionMessage(ex)); }

          finally {
          if (s != null)

          { s.close(); }

          }
          }
          // Check table again
          isok = checkTable(tablename);
          if (!isok)

          { throw new SQLException("Finally failed to repair table " + tablename); }

          else

          { _log.log( Level.INFO, _logprefix + "Table " + tablename + " repaired successfully"); }

          }

          }

          private boolean compressTable(String tablename) throws SQLException {
          // Try to compress first
          boolean result = false;
          try {
          CallableStatement cs = _dbconnection_init.prepareCall("CALL SYSCS_UTIL.SYSCS_COMPRESS_TABLE(?, ?, ?)");
          cs.setString(1, "APP");
          cs.setString(2, tablename);
          cs.setShort(3, (short) 1);
          cs.execute();
          result = true;
          int rows = 0;
          String errmsg = null;
          try {
          Statement s = _dbconnection_init.createStatement();
          ResultSet rs = s.executeQuery("SELECT COUNT FROM " + tablename);
          if (rs.next())

          { rows = rs.getInt(1); }

          s.close();
          } catch (Exception ex)

          { // Error errmsg = Utils.getExceptionMessage(ex); }

          _log.log(
          Level.INFO,
          _logprefix + "Compressing table " + tablename + " successful, table with " + (errmsg != null ? errmsg : rows) + " row(s) repaired");
          } catch (SQLException ex)

          { _log.log( Level.INFO, _logprefix + "Compressing table " + tablename + " failed: " + Utils.getExceptionMessage(ex)); }

          finally

          { _dbconnection_init.commit(); }

          return result;
          }

          private boolean checkTable(String tablename) throws SQLException {
          // Check table
          boolean checktableok = false;
          Statement s = null;
          ResultSet rs = null;
          try {
          s = _dbconnection_init.createStatement();
          boolean ischeckresultrs = s.execute("VALUES SYSCS_UTIL.SYSCS_CHECK_TABLE('APP', '" + tablename + "')");
          if (ischeckresultrs) {
          rs = s.getResultSet();
          String[] columnnames = null;
          String[] columnvalues = null;
          while (rs.next()) {
          if (columnnames == null) {
          columnnames = new String[rs.getMetaData().getColumnCount()];
          columnvalues = new String[rs.getMetaData().getColumnCount()];
          for (int c = 0; c < columnnames.length; c++)

          { columnnames[c] = rs.getMetaData().getColumnName(c + 1); columnvalues[c] = rs.getString(c + 1); }

          }
          }
          checktableok =
          columnnames != null && columnnames.length == 1
          && columnvalues != null & columnvalues.length == 1
          && columnnames[0].equals("1")
          && columnvalues[0].equals("1");
          if (!checktableok)

          { // try to repair table _log.log( Level.INFO, _logprefix + "Table " + tablename + " check failed: " + columnvalues[0]); }

          } else

          { _log.log( Level.FINE, _logprefix + "Table " + tablename + " checked"); }

          } catch (Exception ex)

          { _log.log( Level.INFO, _logprefix + "Table " + tablename + " check failed: " + Utils.getExceptionMessage(ex)); }

          finally {
          if (rs != null)

          { rs.close(); }

          if (s != null)

          { s.close(); }

          _dbconnection_init.commit();
          }
          return checktableok;
          }

          Show
          SCHAEFER B.S. added a comment - Dear Mike, thanks for your mail, but let me point out the tedious thing: In our case the error occured late after startup on inserting records. A call to CALL SYSCS_UTIL.SYSCS_COMPRESS_TABLE(?, ?, ?) solved the problem In the past we had other erros like "The conglomerate does not exist" due to virus scans or other spooky/magic things. To put in a nutshell: Derby can do a "self healing" itself!!! I paste below the code we add as "bootstrap" after database startup to make sure, everything is allright! Tedious, isn't? Furthermore: We insert dummy records into all tables and deleting them immediately to find the errors like XBM0U. My suggestion is: Put that "self healing" code as OPTION into database startup code (bootstrap) or in SQLException when it occurs to try a self repair If you are not convinced or not agree, I don't mind -> we implemented that code for robustness Regards Andreas ------------------------------------------- private void checkAndRepairTable(String tablename) throws SQLException { boolean isok = checkTable(tablename); if (!isok) { _log.log( Level.INFO, _logprefix + "Trying to repair table " + tablename + " ..."); isok = compressTable(tablename); if (!isok) { // Rename table, delete indexes and create new Statement s = null; try { String renamed_table = tablename + "_DAMAGED"; boolean docopy = true; try { renameTable(tablename, renamed_table); } catch (SQLException ex) { //If data can't be read anymore, droptable! _log.log( Level.INFO, _logprefix + "Repair table " + tablename + " failed, table can't be renamed: " + Utils.getExceptionMessage(ex)); dropTable(tablename); } _dbconnection_init.commit(); // Drop indexes dropTableIndexes(renamed_table); _dbconnection_init.commit(); s = _dbconnection_init.createStatement(); // Drop org table and create it again dropTable(tablename); createTable(s, tablename); if (docopy) { try { copyTable(renamed_table, tablename); dropTable(renamed_table); } catch (SQLException ex) { _log.log( Level.WARNING, _logprefix + "Failed to repair table " + tablename + " : Could not copy data, error was : " + Utils.getExceptionMessage(ex, true)); } } } catch (Exception ex) { _log.log( Level.WARNING, _logprefix + "Failed to repair table " + tablename + " : " + Utils.getExceptionMessage(ex)); } finally { if (s != null) { s.close(); } } } // Check table again isok = checkTable(tablename); if (!isok) { throw new SQLException("Finally failed to repair table " + tablename); } else { _log.log( Level.INFO, _logprefix + "Table " + tablename + " repaired successfully"); } } } private boolean compressTable(String tablename) throws SQLException { // Try to compress first boolean result = false; try { CallableStatement cs = _dbconnection_init.prepareCall("CALL SYSCS_UTIL.SYSCS_COMPRESS_TABLE(?, ?, ?)"); cs.setString(1, "APP"); cs.setString(2, tablename); cs.setShort(3, (short) 1); cs.execute(); result = true; int rows = 0; String errmsg = null; try { Statement s = _dbconnection_init.createStatement(); ResultSet rs = s.executeQuery("SELECT COUNT FROM " + tablename); if (rs.next()) { rows = rs.getInt(1); } s.close(); } catch (Exception ex) { // Error errmsg = Utils.getExceptionMessage(ex); } _log.log( Level.INFO, _logprefix + "Compressing table " + tablename + " successful, table with " + (errmsg != null ? errmsg : rows) + " row(s) repaired"); } catch (SQLException ex) { _log.log( Level.INFO, _logprefix + "Compressing table " + tablename + " failed: " + Utils.getExceptionMessage(ex)); } finally { _dbconnection_init.commit(); } return result; } private boolean checkTable(String tablename) throws SQLException { // Check table boolean checktableok = false; Statement s = null; ResultSet rs = null; try { s = _dbconnection_init.createStatement(); boolean ischeckresultrs = s.execute("VALUES SYSCS_UTIL.SYSCS_CHECK_TABLE('APP', '" + tablename + "')"); if (ischeckresultrs) { rs = s.getResultSet(); String[] columnnames = null; String[] columnvalues = null; while (rs.next()) { if (columnnames == null) { columnnames = new String [rs.getMetaData().getColumnCount()] ; columnvalues = new String [rs.getMetaData().getColumnCount()] ; for (int c = 0; c < columnnames.length; c++) { columnnames[c] = rs.getMetaData().getColumnName(c + 1); columnvalues[c] = rs.getString(c + 1); } } } checktableok = columnnames != null && columnnames.length == 1 && columnvalues != null & columnvalues.length == 1 && columnnames [0] .equals("1") && columnvalues [0] .equals("1"); if (!checktableok) { // try to repair table _log.log( Level.INFO, _logprefix + "Table " + tablename + " check failed: " + columnvalues[0]); } } else { _log.log( Level.FINE, _logprefix + "Table " + tablename + " checked"); } } catch (Exception ex) { _log.log( Level.INFO, _logprefix + "Table " + tablename + " check failed: " + Utils.getExceptionMessage(ex)); } finally { if (rs != null) { rs.close(); } if (s != null) { s.close(); } _dbconnection_init.commit(); } return checktableok; }
          Hide
          Mike Matrigali added a comment - - edited

          Andreas, I just wanted to make you aware that Derby currently does not provide the functionality you are
          looking for. Functionality to recover without the ability to write things to disk in a reliable manner would
          be great, but does not exist for Derby. Derby has a few tools to recognize some obvious problems, and some
          tools work to fix the obvious ones if you happen to be lucky enough to break a user index rather than a base
          table or a system catalog.

          Any appearance that Derby can properly function in a no-disk sync environment is an accident
          and all utilities that Derby currently have can not guarantee proper recovery in this case. The current
          design of Derby depends on knowing when bytes written to disk actually have hit the disk. At any time in
          the future, while running with write cache enabled, following a machine crash you may get a Derby error that causes your entire
          database to not be bootable and the only option at that point would be to recover from a valid backup (and note
          our backup strategy also requires that bytes actually hit the disk, which can't be guaranteed in a write cache
          environment so your backups might be bad also). These errors might show up as bad checksums in log or disk, or log recovery errors
          where the bytes in the middle of the recovery log have not hit disk and/or written out of order, or bad data in user data not
          recognized by the conistency checker.

          Note Derby has no problem with any amount of caching in general, in fact it benefits highly from OS caches on reads, it just depends on
          the java functionality that says when we ask to wait for a write to hit disk to not return until it has. Derby currently has no answer
          to environments that cannot guarantee disk sync when specifically requested.

          Show
          Mike Matrigali added a comment - - edited Andreas, I just wanted to make you aware that Derby currently does not provide the functionality you are looking for. Functionality to recover without the ability to write things to disk in a reliable manner would be great, but does not exist for Derby. Derby has a few tools to recognize some obvious problems, and some tools work to fix the obvious ones if you happen to be lucky enough to break a user index rather than a base table or a system catalog. Any appearance that Derby can properly function in a no-disk sync environment is an accident and all utilities that Derby currently have can not guarantee proper recovery in this case. The current design of Derby depends on knowing when bytes written to disk actually have hit the disk. At any time in the future, while running with write cache enabled, following a machine crash you may get a Derby error that causes your entire database to not be bootable and the only option at that point would be to recover from a valid backup (and note our backup strategy also requires that bytes actually hit the disk, which can't be guaranteed in a write cache environment so your backups might be bad also). These errors might show up as bad checksums in log or disk, or log recovery errors where the bytes in the middle of the recovery log have not hit disk and/or written out of order, or bad data in user data not recognized by the conistency checker. Note Derby has no problem with any amount of caching in general, in fact it benefits highly from OS caches on reads, it just depends on the java functionality that says when we ask to wait for a write to hit disk to not return until it has. Derby currently has no answer to environments that cannot guarantee disk sync when specifically requested.
          Hide
          SCHAEFER B.S. added a comment -

          Dear Mike,

          I completely understand the problem, but I would recommend another "philosophy": Expect disc crashes as the world's realtiy -> Let the developer choose/select, what its requirements is.

          1) In our case, a logging table was corrupted and the application stopped, as we had implemented an database check on startup, but it did not fail. It fails on inserting the next record into the table and stopped the application.
          This is really bad for us, but for others its maybe OK, as they want to recover the data manually or by the best method.

          2) For us it is nearly unimportant if we are loosing a few records of millions, but the database is working reliable!
          Additonal: We have tables with important records, which are checked by the application itself, if they are corrupted or safe, and others like logs, which are "unimportant".

          3) In enterprise environments, especially in virtual infrastructures, you will have no chance to disable all caches and guarantee disc sync, i.e. the OS caches, the hypervisor (vmware, hyper-v, xen, etc.) caches, the SAN caches, and the RAID controller also caches.

          So I would prefer to let the developer select upon his needs:

          • Startup fails
          • Startup with automatic recovery in different modes (expensive, inexpensive),
          • Automatic recovery/repair, if it occurs -> in our example: If the insert fails to a table, try to "auto-repair" it, maybe by a system property like -Dapache.derby.autorepair = ....

          Thanks a lot at all!

          Andreas

          Show
          SCHAEFER B.S. added a comment - Dear Mike, I completely understand the problem, but I would recommend another "philosophy": Expect disc crashes as the world's realtiy -> Let the developer choose/select, what its requirements is. 1) In our case, a logging table was corrupted and the application stopped, as we had implemented an database check on startup, but it did not fail. It fails on inserting the next record into the table and stopped the application. This is really bad for us, but for others its maybe OK, as they want to recover the data manually or by the best method. 2) For us it is nearly unimportant if we are loosing a few records of millions, but the database is working reliable! Additonal: We have tables with important records, which are checked by the application itself, if they are corrupted or safe, and others like logs, which are "unimportant". 3) In enterprise environments, especially in virtual infrastructures, you will have no chance to disable all caches and guarantee disc sync, i.e. the OS caches, the hypervisor (vmware, hyper-v, xen, etc.) caches, the SAN caches, and the RAID controller also caches. So I would prefer to let the developer select upon his needs: Startup fails Startup with automatic recovery in different modes (expensive, inexpensive), Automatic recovery/repair, if it occurs -> in our example: If the insert fails to a table, try to "auto-repair" it, maybe by a system property like -Dapache.derby.autorepair = .... Thanks a lot at all! Andreas
          Hide
          SCHAEFER B.S. added a comment -

          Thanks Rick! Habe a nice weekend!

          Regards

          Andreas

          Show
          SCHAEFER B.S. added a comment - Thanks Rick! Habe a nice weekend! Regards Andreas
          Hide
          Mike Matrigali added a comment -

          It should be noted that running and passing the consistency checker does not guarantee that this database is consistent
          and recovered fully.

          For instance because the transaction log has been compromised by running with write cache on and the machine crashed
          the users data may not be transactionally consistent. Data from committed transactions may be totally missing or partial
          changes may exist. There is no way for the system to check for this.

          Derby provides database recovery and transactional consistency only if when it asks the OS to sync data to disk the OS does it. There
          are not tools that guarantee complete recovery and transactional consistency if write caching is enabled. Tools like the consistency
          checker can catch a small portion of the problems and find them up front rather than wait for the application to hit them - but they are
          not complete and likely never can be.

          Show
          Mike Matrigali added a comment - It should be noted that running and passing the consistency checker does not guarantee that this database is consistent and recovered fully. For instance because the transaction log has been compromised by running with write cache on and the machine crashed the users data may not be transactionally consistent. Data from committed transactions may be totally missing or partial changes may exist. There is no way for the system to check for this. Derby provides database recovery and transactional consistency only if when it asks the OS to sync data to disk the OS does it. There are not tools that guarantee complete recovery and transactional consistency if write caching is enabled. Tools like the consistency checker can catch a small portion of the problems and find them up front rather than wait for the application to hit them - but they are not complete and likely never can be.
          Hide
          Rick Hillegas added a comment -

          Thanks, Andreas. I have logged DERBY-6327 and DERBY-6328 to track the enhancement requests which have come out of this discussion. Thanks.

          Show
          Rick Hillegas added a comment - Thanks, Andreas. I have logged DERBY-6327 and DERBY-6328 to track the enhancement requests which have come out of this discussion. Thanks.
          Hide
          SCHAEFER B.S. added a comment -

          Dear Rick,

          thanks for the long explanation. Some notes:

          I would be good to give that information to the developer and let him decide what do, because it is widely application dependent, if a consistency check or a full scan is necessary or not, i.e.

          • a bank account needs transaction safety (-> complete scan)
          • a web server log is absolutely uninteressting ( -> just repair quick)

          Another idea is to put that in general variable or connection property, like -Dapache.derby.healthchecklevel=[all, conglomerates, never] ... then developer can decide also, what to do ...

          Regards Andreas

          Show
          SCHAEFER B.S. added a comment - Dear Rick, thanks for the long explanation. Some notes: I would be good to give that information to the developer and let him decide what do, because it is widely application dependent, if a consistency check or a full scan is necessary or not, i.e. a bank account needs transaction safety (-> complete scan) a web server log is absolutely uninteressting ( -> just repair quick) Another idea is to put that in general variable or connection property, like -Dapache.derby.healthchecklevel= [all, conglomerates, never] ... then developer can decide also, what to do ... Regards Andreas
          Hide
          Rick Hillegas added a comment -

          Hi Andreas,

          What you suggest is an interesting idea. A slightly more refined version of it might be: If the database is not shut down gracefully, then Derby could check the integrity of all the conglomerates after performing crash recovery. On the small data set you sent me, that would not excessively slow down the reboot of the database. We would need to be aware of the following issues:

          1) This would probably dramatically slow down the reboot of large databases. Recent activity on the Derby mailing lists suggests that many Derby-powered applications operate on datasets which are hundreds of gigabytes in size. The user experience probably wouldn't be as bad as what happens when the operating system checks the whole file system on the first reboot after a machine crash. But the delay would be noticeable.

          2) Derby can't tell the difference between an application crash and a machine crash. Complex, crash-prone applications would have to pay this reboot tax often.

          This kind of change to Derby behavior would probably be considered a backward incompatibility which applications would have to opt into.

          In addition, the error message your customer received is probably something we could improve. At first blush it seems that Derby ought to be able to tell the user more about which conglomerate is corrupt. We could certainly improve our repair tools so that the information from that error message could be passed to a tool which would drop and recreate the broken index. The tool would have to be clever enough to handle foreign key dependencies.

          Thanks,
          -Rick

          Show
          Rick Hillegas added a comment - Hi Andreas, What you suggest is an interesting idea. A slightly more refined version of it might be: If the database is not shut down gracefully, then Derby could check the integrity of all the conglomerates after performing crash recovery. On the small data set you sent me, that would not excessively slow down the reboot of the database. We would need to be aware of the following issues: 1) This would probably dramatically slow down the reboot of large databases. Recent activity on the Derby mailing lists suggests that many Derby-powered applications operate on datasets which are hundreds of gigabytes in size. The user experience probably wouldn't be as bad as what happens when the operating system checks the whole file system on the first reboot after a machine crash. But the delay would be noticeable. 2) Derby can't tell the difference between an application crash and a machine crash. Complex, crash-prone applications would have to pay this reboot tax often. This kind of change to Derby behavior would probably be considered a backward incompatibility which applications would have to opt into. In addition, the error message your customer received is probably something we could improve. At first blush it seems that Derby ought to be able to tell the user more about which conglomerate is corrupt. We could certainly improve our repair tools so that the information from that error message could be passed to a tool which would drop and recreate the broken index. The tool would have to be clever enough to handle foreign key dependencies. Thanks, -Rick
          Hide
          SCHAEFER B.S. added a comment -

          Dear Rick,

          the most important thing would be an autorepair function -> It doesn't matter (from my point of view) if there are records missing, if disc caching is enabled.
          But the database should start up and should work.

          In this case, database started up, schema check was successful (getMetaData()), but the error occured late on inserting new records.

          Is it possible to check this corruptness at startup??

          Thank you!

          Regards

          Andreas

          Show
          SCHAEFER B.S. added a comment - Dear Rick, the most important thing would be an autorepair function -> It doesn't matter (from my point of view) if there are records missing, if disc caching is enabled. But the database should start up and should work. In this case, database started up, schema check was successful (getMetaData()), but the error occured late on inserting new records. Is it possible to check this corruptness at startup?? Thank you! Regards Andreas
          Hide
          Rick Hillegas added a comment -

          Thanks Andreas. Enabling the write-cache will boost your customer's performance but it can result in corrupted data if the machine crashes. Thanks.

          Show
          Rick Hillegas added a comment - Thanks Andreas. Enabling the write-cache will boost your customer's performance but it can result in corrupted data if the machine crashes. Thanks.
          Hide
          SCHAEFER B.S. added a comment -

          Dear Rick,

          thank you for your support!

          to 1) No!

          to 2) It was a system of a customer -> I think yes

          to 3) Yes, another application crashed down Windows at all -> good job!

          I repaired the table by compressing them, as commeted at JIRA

          Sorry, for the "put 'em in a nutshell" answers, but I'm under pressure at the momment, thank you!

          Regards Andreas

          Show
          SCHAEFER B.S. added a comment - Dear Rick, thank you for your support! to 1) No! to 2) It was a system of a customer -> I think yes to 3) Yes, another application crashed down Windows at all -> good job! I repaired the table by compressing them, as commeted at JIRA Sorry, for the "put 'em in a nutshell" answers, but I'm under pressure at the momment, thank you! Regards Andreas
          Hide
          Rick Hillegas added a comment -

          Hi Andreas,

          Thanks for sending me the corrupted database. The good news is that it's still bootable and the system tables seem to be intact. When I run the SYSCS_UTIL.SYSCS_CHECK_TABLE utility and then drill down with queries against SYS.SYSTABLES, SYS.SYSCONGLOMERATES, and SYS.SYSCONSTRAINTS, I see that the corruption is isolated to the index which backs the primary key on the APP.TBL_LOGGING table. After dropping that primary key, the SYSCS_UTIL.SYSCS_CHECK_TABLE runs cleanly:

          alter table TBL_LOGGING drop primary key;

          SELECT schemaname, tablename,
          SYSCS_UTIL.SYSCS_CHECK_TABLE(schemaname, tablename)
          FROM sys.sysschemas s, sys.systables t
          WHERE s.schemaid = t.schemaid;

          So I think you should be able to repair this database by dropping and then re-creating the primary key on TBL_LOGGING.

          I will see if I can figure out anything more about this corruption by examining the broken index. It is not clear yet whether the index was corrupted by the btree split or whether the btree split simply tripped across a corruption which happened earlier. I may be able to push this forward. But it is also possible that the original problem can't be reconstructed.

          A couple more questions for you (basically the same list of questions further up on this issue):

          1) Do you compress database tables?

          2) Is the disk write-cache enabled or disabled?

          3) Did anything unusual happen before the corruption. E.g., an application or system crash?

          Thanks,
          -Rick

          Show
          Rick Hillegas added a comment - Hi Andreas, Thanks for sending me the corrupted database. The good news is that it's still bootable and the system tables seem to be intact. When I run the SYSCS_UTIL.SYSCS_CHECK_TABLE utility and then drill down with queries against SYS.SYSTABLES, SYS.SYSCONGLOMERATES, and SYS.SYSCONSTRAINTS, I see that the corruption is isolated to the index which backs the primary key on the APP.TBL_LOGGING table. After dropping that primary key, the SYSCS_UTIL.SYSCS_CHECK_TABLE runs cleanly: alter table TBL_LOGGING drop primary key; SELECT schemaname, tablename, SYSCS_UTIL.SYSCS_CHECK_TABLE(schemaname, tablename) FROM sys.sysschemas s, sys.systables t WHERE s.schemaid = t.schemaid; So I think you should be able to repair this database by dropping and then re-creating the primary key on TBL_LOGGING. I will see if I can figure out anything more about this corruption by examining the broken index. It is not clear yet whether the index was corrupted by the btree split or whether the btree split simply tripped across a corruption which happened earlier. I may be able to push this forward. But it is also possible that the original problem can't be reconstructed. A couple more questions for you (basically the same list of questions further up on this issue): 1) Do you compress database tables? 2) Is the disk write-cache enabled or disabled? 3) Did anything unusual happen before the corruption. E.g., an application or system crash? Thanks, -Rick
          Hide
          Rick Hillegas added a comment -

          Kathey raises a good point. What version of Derby are you using? Thanks.

          Show
          Rick Hillegas added a comment - Kathey raises a good point. What version of Derby are you using? Thanks.
          Hide
          SCHAEFER B.S. added a comment -

          Some additional notes:

          • Table could be repaired by CALL SYSCS_UTIL.SYSCS_COMPRESS_TABLE(?, ?, ?)
          • Used version is: ApacheDB 10.9.1.0
          • Sending the archive DB to Rick
          Show
          SCHAEFER B.S. added a comment - Some additional notes: Table could be repaired by CALL SYSCS_UTIL.SYSCS_COMPRESS_TABLE(?, ?, ?) Used version is: ApacheDB 10.9.1.0 Sending the archive DB to Rick
          Hide
          Rick Hillegas added a comment -

          You could send the archive to my Apache account (rhillegas at apache dot org). I think the risk is very low that Apache engineers would violate your confidentiality.

          Show
          Rick Hillegas added a comment - You could send the archive to my Apache account (rhillegas at apache dot org). I think the risk is very low that Apache engineers would violate your confidentiality.
          Hide
          Kathey Marsden added a comment -

          Might DERBY-5258 be relevant. I think can cause corruption during a btree split. I notice the user version 10.5.3.0 doesn't have the fix.

          Show
          Kathey Marsden added a comment - Might DERBY-5258 be relevant. I think can cause corruption during a btree split. I notice the user version 10.5.3.0 doesn't have the fix.
          Hide
          SCHAEFER B.S. added a comment -

          Archive size (RAR) is about 1,8 MB, but content is confidental. Is there a possibility to send you this archive directly or this is upload for developers only?

          Show
          SCHAEFER B.S. added a comment - Archive size (RAR) is about 1,8 MB, but content is confidental. Is there a possibility to send you this archive directly or this is upload for developers only?
          Hide
          Rick Hillegas added a comment -

          If the archive is small enough, you could attach it to this issue. Thanks.

          Show
          Rick Hillegas added a comment - If the archive is small enough, you could attach it to this issue. Thanks.
          Hide
          SCHAEFER B.S. added a comment -

          Note: Stacktrace above is from a windows system

          Show
          SCHAEFER B.S. added a comment - Note: Stacktrace above is from a windows system
          Hide
          SCHAEFER B.S. added a comment -

          Do you like to have the DB? I could prepare a RAR archive -> where to send?

          Show
          SCHAEFER B.S. added a comment - Do you like to have the DB? I could prepare a RAR archive -> where to send?
          Hide
          Rick Hillegas added a comment -

          Thanks for the new stack trace. It looks to me like a data corruption during a BTree split.

          Show
          Rick Hillegas added a comment - Thanks for the new stack trace. It looks to me like a data corruption during a BTree split.
          Hide
          SCHAEFER B.S. added a comment -

          Found it TODAY after a system crash

          WARNING: 28.08.2013 20:03:38.936 : LogPublisher for handler ArchiveLogDBHandler for 2013-08_857845555 had error : No class was registered for identifier 0. -> No class was registered for identifier 0. -> No class was registered for identifier 0. -> No class was registered for identifier 0. -> No class was registered for identifier 0. -> No class was registered for identifier 0. -> No class was registered for identifier 0.
          Nested exception is: No class was registered for identifier 0.
          java.sql.BatchUpdateException: No class was registered for identifier 0.
          at org.apache.derby.impl.jdbc.EmbedStatement.executeBatch(EmbedStatement.java:1008)
          at com.sbs.storage.MultiConnectionDerbyArchiveDatabase.addLogEntry(MultiConnectionDerbyArchiveDatabase.java:2314)
          at com.sbs.storage.ArchiveLogDBHandler.publish(ArchiveLogDBHandler.java:40)
          at com.sbs.utils.logging.LogManager$LogPublisher.run(LogManager.java:521)
          Caused by: java.sql.SQLException: No class was registered for identifier 0.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:98)
          at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Util.java:256)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:424)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:353)
          at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2360)
          at org.apache.derby.impl.jdbc.ConnectionChild.handleException(ConnectionChild.java:82)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1334)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeBatchElement(EmbedPreparedStatement.java:1067)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeBatch(EmbedStatement.java:984)
          ... 3 more
          Caused by: java.sql.SQLException: No class was registered for identifier 0.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:42)
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:122)
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71)
          ... 11 more
          Caused by: ERROR XBM0U: No class was registered for identifier 0.
          at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:278)
          at org.apache.derby.impl.services.monitor.BaseMonitor.classFromIdentifier(BaseMonitor.java:650)
          at org.apache.derby.impl.services.monitor.BaseMonitor.newInstanceFromIdentifier(BaseMonitor.java:663)
          at org.apache.derby.iapi.services.monitor.Monitor.newInstanceFromIdentifier(Monitor.java:643)
          at org.apache.derby.impl.store.access.btree.ControlRow.getControlRowForPage(ControlRow.java:905)
          at org.apache.derby.impl.store.access.btree.ControlRow.get(ControlRow.java:844)
          at org.apache.derby.impl.store.access.btree.ControlRow.get(ControlRow.java:820)
          at org.apache.derby.impl.store.access.btree.ControlRow.getRightSibling(ControlRow.java:531)
          at org.apache.derby.impl.store.access.btree.ControlRow.linkRight(ControlRow.java:1678)
          at org.apache.derby.impl.store.access.btree.LeafControlRow.splitFor(LeafControlRow.java:651)
          at org.apache.derby.impl.store.access.btree.BranchControlRow.splitFor(BranchControlRow.java:906)
          at org.apache.derby.impl.store.access.btree.BTreeController.start_xact_and_dosplit(BTreeController.java:368)
          at org.apache.derby.impl.store.access.btree.BTreeController.doIns(BTreeController.java:1014)
          at org.apache.derby.impl.store.access.btree.BTreeController.insert(BTreeController.java:1373)
          at org.apache.derby.impl.store.access.btree.index.B2IController.insert(B2IController.java:210)
          at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(IndexChanger.java:440)
          at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(IndexChanger.java:383)
          at org.apache.derby.impl.sql.execute.IndexChanger.insert(IndexChanger.java:590)
          at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(IndexSetChanger.java:268)
          at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(RowChangerImpl.java:453)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(InsertResultSet.java:1048)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(InsertResultSet.java:508)
          at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(GenericPreparedStatement.java:443)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:324)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1242)
          ... 5 mor

          Show
          SCHAEFER B.S. added a comment - Found it TODAY after a system crash WARNING: 28.08.2013 20:03:38.936 : LogPublisher for handler ArchiveLogDBHandler for 2013-08_857845555 had error : No class was registered for identifier 0. -> No class was registered for identifier 0. -> No class was registered for identifier 0. -> No class was registered for identifier 0. -> No class was registered for identifier 0. -> No class was registered for identifier 0. -> No class was registered for identifier 0. Nested exception is: No class was registered for identifier 0. java.sql.BatchUpdateException: No class was registered for identifier 0. at org.apache.derby.impl.jdbc.EmbedStatement.executeBatch(EmbedStatement.java:1008) at com.sbs.storage.MultiConnectionDerbyArchiveDatabase.addLogEntry(MultiConnectionDerbyArchiveDatabase.java:2314) at com.sbs.storage.ArchiveLogDBHandler.publish(ArchiveLogDBHandler.java:40) at com.sbs.utils.logging.LogManager$LogPublisher.run(LogManager.java:521) Caused by: java.sql.SQLException: No class was registered for identifier 0. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:98) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Util.java:256) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:424) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:353) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2360) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(ConnectionChild.java:82) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1334) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeBatchElement(EmbedPreparedStatement.java:1067) at org.apache.derby.impl.jdbc.EmbedStatement.executeBatch(EmbedStatement.java:984) ... 3 more Caused by: java.sql.SQLException: No class was registered for identifier 0. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:42) at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:122) at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71) ... 11 more Caused by: ERROR XBM0U: No class was registered for identifier 0. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:278) at org.apache.derby.impl.services.monitor.BaseMonitor.classFromIdentifier(BaseMonitor.java:650) at org.apache.derby.impl.services.monitor.BaseMonitor.newInstanceFromIdentifier(BaseMonitor.java:663) at org.apache.derby.iapi.services.monitor.Monitor.newInstanceFromIdentifier(Monitor.java:643) at org.apache.derby.impl.store.access.btree.ControlRow.getControlRowForPage(ControlRow.java:905) at org.apache.derby.impl.store.access.btree.ControlRow.get(ControlRow.java:844) at org.apache.derby.impl.store.access.btree.ControlRow.get(ControlRow.java:820) at org.apache.derby.impl.store.access.btree.ControlRow.getRightSibling(ControlRow.java:531) at org.apache.derby.impl.store.access.btree.ControlRow.linkRight(ControlRow.java:1678) at org.apache.derby.impl.store.access.btree.LeafControlRow.splitFor(LeafControlRow.java:651) at org.apache.derby.impl.store.access.btree.BranchControlRow.splitFor(BranchControlRow.java:906) at org.apache.derby.impl.store.access.btree.BTreeController.start_xact_and_dosplit(BTreeController.java:368) at org.apache.derby.impl.store.access.btree.BTreeController.doIns(BTreeController.java:1014) at org.apache.derby.impl.store.access.btree.BTreeController.insert(BTreeController.java:1373) at org.apache.derby.impl.store.access.btree.index.B2IController.insert(B2IController.java:210) at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(IndexChanger.java:440) at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(IndexChanger.java:383) at org.apache.derby.impl.sql.execute.IndexChanger.insert(IndexChanger.java:590) at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(IndexSetChanger.java:268) at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(RowChangerImpl.java:453) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(InsertResultSet.java:1048) at org.apache.derby.impl.sql.execute.InsertResultSet.open(InsertResultSet.java:508) at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(GenericPreparedStatement.java:443) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:324) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1242) ... 5 mor
          Hide
          Kathey Marsden added a comment -

          We have not heard back in over a year on this issue, perhaps we should close it CannotReproduce and it can be reopened if kw has a reproduction or information.

          Show
          Kathey Marsden added a comment - We have not heard back in over a year on this issue, perhaps we should close it CannotReproduce and it can be reopened if kw has a reproduction or information.
          Hide
          Rick Hillegas added a comment -

          Hi kw,

          It seems that you have seen this error at least twice, with a couple month gap between the occurrences. Some more questions:

          1) Do you compress database tables?

          2) Is the disk write-cache enabled or disabled?

          3) Does anything unusual happen before the corruptions. E.g., an application or system crash?

          Thanks,
          -Rick

          Show
          Rick Hillegas added a comment - Hi kw, It seems that you have seen this error at least twice, with a couple month gap between the occurrences. Some more questions: 1) Do you compress database tables? 2) Is the disk write-cache enabled or disabled? 3) Does anything unusual happen before the corruptions. E.g., an application or system crash? Thanks, -Rick
          Hide
          Mike Matrigali added a comment -

          10.9 derby triage.

          Show
          Mike Matrigali added a comment - 10.9 derby triage.
          Hide
          kw added a comment -

          I reminded myself that before restore Artifactory I did backup logs. I don't see OverlappingFileLockException in derby.log, but I see "ClassNotFoundException: ERROR XBM0U":

          ------------ BEGIN SHUTDOWN ERROR STACK -------------

          ERROR XSLA3: Log Corrupted, has invalid data in the log stream.
          (...)
          Caused by: java.lang.ClassNotFoundException: ERROR XBM0U: No class was registered for identifier 5.
          (...)
          ============= begin nested exception, level (1) ===========
          java.lang.ClassNotFoundException: ERROR XBM0U: No class was registered for identifier 5.
          (...)
          ============= end nested exception, level (1) ===========

          ------------ END SHUTDOWN ERROR STACK -------------

          Show
          kw added a comment - I reminded myself that before restore Artifactory I did backup logs. I don't see OverlappingFileLockException in derby.log, but I see "ClassNotFoundException: ERROR XBM0U": ------------ BEGIN SHUTDOWN ERROR STACK ------------- ERROR XSLA3: Log Corrupted, has invalid data in the log stream. (...) Caused by: java.lang.ClassNotFoundException: ERROR XBM0U: No class was registered for identifier 5. (...) ============= begin nested exception, level (1) =========== java.lang.ClassNotFoundException: ERROR XBM0U: No class was registered for identifier 5. (...) ============= end nested exception, level (1) =========== ------------ END SHUTDOWN ERROR STACK -------------
          Hide
          kw added a comment -

          Unfortunately, I restarted Artifactory and derby.log was rotated and I am not able to tell that.

          Show
          kw added a comment - Unfortunately, I restarted Artifactory and derby.log was rotated and I am not able to tell that.
          Hide
          Rick Hillegas added a comment -

          Thanks, kw. The original derby.log showed a java.nio.channels.OverlappingFileLockException. Do you see one of those exceptions in derby.log around this new recurrence of the problem? Thanks.

          Show
          Rick Hillegas added a comment - Thanks, kw. The original derby.log showed a java.nio.channels.OverlappingFileLockException. Do you see one of those exceptions in derby.log around this new recurrence of the problem? Thanks.
          Hide
          kw added a comment -

          No, not tried. Why do you think that it would help, since the de facto start was that the same base.

          As a workaround I created Artifactory again. Unfortunately the problem after a few months to repeat. Do you already know what is causing and how to avoid a repetition of this problem?

          Show
          kw added a comment - No, not tried. Why do you think that it would help, since the de facto start was that the same base. As a workaround I created Artifactory again. Unfortunately the problem after a few months to repeat. Do you already know what is causing and how to avoid a repetition of this problem?
          Hide
          Rick Hillegas added a comment -

          Hi kw,

          Have you tried copying the corrupt database and booting the copy? I'm puzzled by the discrepancy between the original stack trace and the ability of LogFileReader to read the log from start to finish.

          Thanks,
          -Rick

          Show
          Rick Hillegas added a comment - Hi kw, Have you tried copying the corrupt database and booting the copy? I'm puzzled by the discrepancy between the original stack trace and the ability of LogFileReader to read the log from start to finish. Thanks, -Rick
          Hide
          Rick Hillegas added a comment -

          I thought I had recorded this comment before, but I guess I didn't:

          The very first error in derby.log is a java.nio.channels.OverlappingFileLockException. This is the exception you see in 10.5.3.0 when two threads in the same VM try to boot a database from different ClassLoaders. That may be a red herring. However, there may be some opportunity for data corruption which we haven't considered when multiple copies of Derby are running in different ClassLoaders.

          Show
          Rick Hillegas added a comment - I thought I had recorded this comment before, but I guess I didn't: The very first error in derby.log is a java.nio.channels.OverlappingFileLockException. This is the exception you see in 10.5.3.0 when two threads in the same VM try to boot a database from different ClassLoaders. That may be a red herring. However, there may be some opportunity for data corruption which we haven't considered when multiple copies of Derby are running in different ClassLoaders.
          Hide
          kw added a comment -

          In resonse to "Are you seeing the same exception when you try to connect to both databases"

          In this case Artifactory connects to this dababases and everything I know is in derby.log file.

          Show
          kw added a comment - In resonse to "Are you seeing the same exception when you try to connect to both databases" In this case Artifactory connects to this dababases and everything I know is in derby.log file.
          Hide
          Rick Hillegas added a comment -

          Thanks for those log directories, kw. Another question: Are you seeing the same exception when you try to connect to both databases?

          I have run the ControlFileReader and LogFileReader tools against the log directories which kw attached (those tools are attached to DERBY-5195). The db database is in a state which I don't understand. I don't see anything obviously strange about the store database, though.

          I also don't see any corrupt log records which would give rise to the original exception which kw attached.

          A question for other Derby developers: Any theories about what is going on in the db database? The log file there does not agree with the control file about when the last checkpoint occurred, and the log file ends abruptly.

          Thanks,
          -Rick

          ----------------------------

          Here is what I see:

          The db database (looks odd):

          1) The log directory contains 3 files: log.ctrl, logmirror.ctrl, and log15105.dat.

          2) The two control files are identical.

          3) The control files assert that the last checkpoint was taken at log instant (15105, 778183).

          4) However, there is no log record for that log instant in log15105.dat and the last CheckpointOperation recorded in log15105.dat occurred before then at ( 15105, 777959 ).

          5) The log file ends abruptly at log instant (15105, 792785 ), a BeginXact record. LogFileReader trips over an EOF exception trying to read the next log record after that.

          The store database (looks ok):

          1) The log directory contains 3 files: log.ctrl, logmirror.ctrl, and log4081.dat.

          2) The two control files are identical.

          3) The control files assert that the last checkpoint was taken at log instant ( 4081, 503839 ).

          4) There is, in fact, a CheckpointOperation at that log instant in log4081.dat and that is the last CheckpointOperation in the file.

          5) LogFileReader successfully deserializes the entire log4081.dat file.

          Show
          Rick Hillegas added a comment - Thanks for those log directories, kw. Another question: Are you seeing the same exception when you try to connect to both databases? I have run the ControlFileReader and LogFileReader tools against the log directories which kw attached (those tools are attached to DERBY-5195 ). The db database is in a state which I don't understand. I don't see anything obviously strange about the store database, though. I also don't see any corrupt log records which would give rise to the original exception which kw attached. A question for other Derby developers: Any theories about what is going on in the db database? The log file there does not agree with the control file about when the last checkpoint occurred, and the log file ends abruptly. Thanks, -Rick ---------------------------- Here is what I see: The db database (looks odd): 1) The log directory contains 3 files: log.ctrl, logmirror.ctrl, and log15105.dat. 2) The two control files are identical. 3) The control files assert that the last checkpoint was taken at log instant (15105, 778183). 4) However, there is no log record for that log instant in log15105.dat and the last CheckpointOperation recorded in log15105.dat occurred before then at ( 15105, 777959 ). 5) The log file ends abruptly at log instant (15105, 792785 ), a BeginXact record. LogFileReader trips over an EOF exception trying to read the next log record after that. The store database (looks ok): 1) The log directory contains 3 files: log.ctrl, logmirror.ctrl, and log4081.dat. 2) The two control files are identical. 3) The control files assert that the last checkpoint was taken at log instant ( 4081, 503839 ). 4) There is, in fact, a CheckpointOperation at that log instant in log4081.dat and that is the last CheckpointOperation in the file. 5) LogFileReader successfully deserializes the entire log4081.dat file.
          Hide
          kw added a comment -

          Logs from store/log.

          Show
          kw added a comment - Logs from store/log.
          Hide
          kw added a comment -

          I check derby.log and I see problem with start two database:

          java.sql.SQLException: Failed to start database '/matrix/hudson/.artifactory/data/store', see the next exception for details.
          java.sql.SQLException: Failed to start database '/matrix/hudson/.artifactory/data/db', see the next exception for details.

          Previously attached logs are from .artifactory/data/db/log

          Show
          kw added a comment - I check derby.log and I see problem with start two database: java.sql.SQLException: Failed to start database '/matrix/hudson/.artifactory/data/store', see the next exception for details. java.sql.SQLException: Failed to start database '/matrix/hudson/.artifactory/data/db', see the next exception for details. Previously attached logs are from .artifactory/data/db/log
          Hide
          kw added a comment -

          Thanks for claryfication. I attach log dir.

          Show
          kw added a comment - Thanks for claryfication. I attach log dir.
          Hide
          Rick Hillegas added a comment -

          Thanks, kw. I'm sorry for not being clear about what additional information I need. A Derby database is a directory containing the following contents. The directory itself has the same name as the database it represents:

          service.properties – This is a control file with information about the database

          seg0 – This subdirectory contains the data files in the database, one file for each heap and index

          log --This subdirectory contains the recovery logs needed to keep Derby in a consistent state after a crash

          What I need is a zip file containing everything in the log subdirectory. Thanks.

          Show
          Rick Hillegas added a comment - Thanks, kw. I'm sorry for not being clear about what additional information I need. A Derby database is a directory containing the following contents. The directory itself has the same name as the database it represents: service.properties – This is a control file with information about the database seg0 – This subdirectory contains the data files in the database, one file for each heap and index log --This subdirectory contains the recovery logs needed to keep Derby in a consistent state after a crash What I need is a zip file containing everything in the log subdirectory. Thanks.
          Hide
          kw added a comment -

          My derby.log.gz.

          Show
          kw added a comment - My derby.log.gz.
          Hide
          Rick Hillegas added a comment -

          This looks like it could be a data corruption to me. The recovery phase is trying to read the log file and dies when it comes to a badly formatted log record. Could you zip up the contents of the log directory so that we can poke around a bit? Thanks.

          Show
          Rick Hillegas added a comment - This looks like it could be a data corruption to me. The recovery phase is trying to read the log file and dies when it comes to a badly formatted log record. Could you zip up the contents of the log directory so that we can poke around a bit? Thanks.

            People

            • Assignee:
              Unassigned
              Reporter:
              kw
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Development