[CONNECTORS-100] DB lock timeout, and/or indefinite or excessive database activity

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: ManifoldCF 0.1, ManifoldCF 0.2, ManifoldCF 0.3, ManifoldCF 0.4
• Fix Version/s: ManifoldCF 0.4
• Component/s: Framework core

Environment

Running unmodified dist/example from trunk/ using the default configuration.

• Assignee: Karl Wright
• Reporter: Andrzej Bialecki
• Created: 07/Sep/10 11:03
• Updated: 02/Nov/11 18:12
• Resolved: 02/Nov/11 18:12

Description

When a job is started and running (via crawler-ui) occasionally it's not possible to display a list of running jobs. The problem persists even after restarting ACF. The following exception is thrown in the console:

org.apache.acf.core.interfaces.ACFException: Database exception: Exception doing query: A lock could not be obtained within the time requested
        at org.apache.acf.core.database.Database.executeViaThread(Database.java:421)
        at org.apache.acf.core.database.Database.executeUncachedQuery(Database.java:465)
        at org.apache.acf.core.database.Database$QueryCacheExecutor.create(Database.java:1072)
        at org.apache.acf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
        at org.apache.acf.core.database.Database.executeQuery(Database.java:167)
        at org.apache.acf.core.database.DBInterfaceDerby.performQuery(DBInterfaceDerby.java:727)
        at org.apache.acf.crawler.jobs.JobManager.makeJobStatus(JobManager.java:5611)
        at org.apache.acf.crawler.jobs.JobManager.getAllStatus(JobManager.java:5549)
        at org.apache.jsp.showjobstatus_jsp._jspService(showjobstatus_jsp.java:316)
        at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:377)
        at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
        at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.sql.SQLTransactionRollbackException: A lock could not be obtained within the time requested
        at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
        at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
        at org.apache.acf.core.database.Database.execute(Database.java:526)
        at org.apache.acf.core.database.Database$ExecuteQueryThread.run(Database.java:381)
Caused by: java.sql.SQLException: A lock could not be obtained within the time requested
        at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
        ... 11 more
Caused by: ERROR 40XL1: A lock could not be obtained within the time requested
        at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
        at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source)
        at org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(Unknown Source)
        at org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(Unknown Source)
        at org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(Unknown Source)
        at org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(Unknown Source)
        at org.apache.derby.impl.store.access.heap.HeapController.lockRow(Unknown Source)
        at org.apache.derby.impl.store.access.heap.HeapController.lockRow(Unknown Source)
        at org.apache.derby.impl.store.access.btree.index.B2IRowLocking3.lockRowOnPage(Unknown Source)
        at org.apache.derby.impl.store.access.btree.index.B2IRowLocking3._lockScanRow(Unknown Source)
        at org.apache.derby.impl.store.access.btree.index.B2IRowLockingRR.lockScanRow(Unknown Source)
        at org.apache.derby.impl.store.access.btree.BTreeForwardScan.fetchRows(Unknown Source)
        at org.apache.derby.impl.store.access.btree.BTreeScan.fetchNextGroup(Unknown Source)
        at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unknown Source)
        at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Unknown Source)
        at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown Source)
        at org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.getRowFromResultSet(Unknown Source)
        at org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.getNextRowFromRS(Unknown Source)
        at org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.loadSorter(Unknown Source)
        at org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.openCore(Unknown Source)
        at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.openCore(Unknown Source)
        at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.open(Unknown Source)
        at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
        at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
        ... 5 more

Issue Links

• is blocked by: DERBY-5142 Optimizer uses table scan when it could use index when multiple OR clauses
• is related to: CONNECTORS-166 Crawl seizes up when running Derby