[CONNECTORS-166] Crawl seizes up when running Derby

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: ManifoldCF 0.1, ManifoldCF 0.2
• Fix Version/s: ManifoldCF 0.2
• Component/s: Framework crawler agent
• Assignee: Karl Wright
• Reporter: Karl Wright
• Created: 27/Feb/11 16:36
• Updated: 16/Mar/11 13:06
• Resolved: 16/Mar/11 13:02

Description

A crawl using multiple worker threads with Derby eventually hangs, because threads get deadlocked dealing with carrydown information. At the time of hang, a thread dump yields:

"Worker thread '5'" daemon prio=6 tid=0x02fc7800 nid=0xd78 in Object.wait() [0x0465f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

• waiting on <0x2858b720> (a org.apache.manifoldcf.core.database.Database$ExecuteQueryThread)
  at java.lang.Thread.join(Unknown Source)
• locked <0x2858b720> (a org.apache.manifoldcf.core.database.Database$ExecuteQueryThread)
  at java.lang.Thread.join(Unknown Source)
  at org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:453)
  at org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:489)
  at org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1131)
  at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
  at org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168)
  at org.apache.manifoldcf.core.database.DBInterfaceDerby.performQuery(DBInterfaceDerby.java:785)
  at org.apache.manifoldcf.crawler.jobs.JobManager.processDeleteHashSet(JobManager.java:2592)
  at org.apache.manifoldcf.crawler.jobs.JobManager.calculateAffectedDeleteCarrydownChildren(JobManager.java:2565)
  at org.apache.manifoldcf.crawler.jobs.JobManager.markDocumentDeletedMultiple(JobManager.java:2494)
  at org.apache.manifoldcf.crawler.system.WorkerThread.processDeleteLists(WorkerThread.java:1077)
  at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:544)

... for at least two threads.

Issue Links

• is blocked by: DERBY-5073 Derby deadlocks without recourse on simultaneous correlated subqueries
• relates to: CONNECTORS-100 DB lock timeout, and/or indefinite or excessive database activity