[CONNECTORS-1156] Can't shut down agents service when automatic ANALYZE TABLE, or any long-running query, is happening

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: ManifoldCF 1.8, ManifoldCF 2.0
• Fix Version/s: ManifoldCF 1.8.2, ManifoldCF 2.0.2, ManifoldCF 1.9, ManifoldCF 2.1
• Component/s: Framework agents process
• Assignee: Karl Wright
• Reporter: Karl Wright
• Created: 04/Feb/15 15:37
• Updated: 16/Feb/15 08:10
• Resolved: 07/Feb/15 23:38

Description

This was detected on MCF 1.8.

The thread blocking shutdown had the following trace:

"Startup thread" daemon prio=10 tid=0x00007fe73012f000 nid=0x340b in Object.wait() [0x00007fe71f7f6000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Thread.join(Thread.java:1281)
    - locked <0x00000000e851a0b8> (a org.apache.manifoldcf.core.database.Database$ExecuteQueryThread)
    at java.lang.Thread.join(Thread.java:1355)
    at org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.finishUp(Database.java:694)
    at org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:728)
    at org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:790)
    at org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1444)
    at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:146)
    at org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:191)
    at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:656)
    at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.analyzeTableInternal(DBInterfacePostgreSQL.java:1431)
    at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.noteModificationsNoTransactions(DBInterfacePostgreSQL.java:1576)
    at org.apache.manifoldcf.core.database.Database.playbackModifications(Database.java:429)
    at org.apache.manifoldcf.core.database.Database.endTransaction(Database.java:414)
    at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.endTransaction(DBInterfacePostgreSQL.java:1231)
    at org.apache.manifoldcf.crawler.jobs.JobManager.resetStartupJob(JobManager.java:7575)
    at org.apache.manifoldcf.crawler.system.StartupThread.run(StartupThread.java:238)

This was after many minutes of waiting for the shutdown to take place, so clearly it's looping in the sense that it keeps starting an ANALYZE, gets interrupted, and retries indefinitely.