[SOLR-2608] TestReplicationHandler is flakey

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: Mark Miller
• Reporter: selckin
• Created: 19/Jun/11 11:20
• Updated: 14/Dec/12 00:41
• Resolved: 14/Dec/12 00:41

Description

I've been running some while(1) tests on trunk, and TestReplicationHandler is very flakey it fails about every 10th run.
Probably not a bug, but the test not waiting correctly

    [junit] Testsuite: org.apache.solr.handler.TestReplicationHandler
    [junit] Testcase: org.apache.solr.handler.TestReplicationHandler:   FAILED
    [junit] ERROR: SolrIndexSearcher opens=48 closes=47
    [junit] junit.framework.AssertionFailedError: ERROR: SolrIndexSearcher opens=48 closes=47
    [junit]     at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:131)
    [junit]     at org.apache.solr.SolrTestCaseJ4.afterClassSolrTestCase(SolrTestCaseJ4.java:74)
    [junit] 
    [junit] 
    [junit] Tests run: 8, Failures: 1, Errors: 0, Time elapsed: 40.772 sec
    [junit] 
    [junit] ------------- Standard Error -----------------
    [junit] 19-Jun-2011 21:26:44 org.apache.solr.handler.SnapPuller fetchLatestIndex
    [junit] SEVERE: Master at: http://localhost:51817/solr/replication is not available. Index fetch failed. Exception: Connection refused
    [junit] 19-Jun-2011 21:26:49 org.apache.solr.common.SolrException log
    [junit] SEVERE: java.util.concurrent.RejectedExecutionException
    [junit]     at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1768)
    [junit]     at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
    [junit]     at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
    [junit]     at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:92)
    [junit]     at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:603)
    [junit]     at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1149)
    [junit]     at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:346)
    [junit]     at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:483)
    [junit]     at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:332)
    [junit]     at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:267)
    [junit]     at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:166)
    [junit]     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    [junit]     at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
    [junit]     at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
    [junit]     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
    [junit]     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
    [junit]     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
    [junit]     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    [junit]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    [junit]     at java.lang.Thread.run(Thread.java:662)
    [junit] 
    [junit] 19-Jun-2011 21:26:51 org.apache.solr.update.SolrIndexWriter finalize
    [junit] SEVERE: SolrIndexWriter was not closed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
    [junit] 19-Jun-2011 21:26:51 org.apache.solr.common.util.ConcurrentLRUCache finalize
    [junit] SEVERE: ConcurrentLRUCache was not destroyed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
    [junit] 19-Jun-2011 21:27:13 org.apache.solr.handler.SnapPuller fetchLatestIndex
    [junit] SEVERE: Master at: http://localhost:46559/solr/replication is not available. Index fetch failed. Exception: Connection refused
    [junit] 19-Jun-2011 21:27:14 org.apache.solr.handler.SnapPuller fetchLatestIndex
    [junit] SEVERE: Master at: http://localhost:46559/solr/replication is not available. Index fetch failed. Exception: Connection refused
    [junit] 19-Jun-2011 21:27:14 org.apache.solr.SolrTestCaseJ4 endTrackingSearchers
    [junit] SEVERE: ERROR: SolrIndexSearcher opens=48 closes=47
    [junit] ------------- ---------------- ---------------
    [junit] TEST org.apache.solr.handler.TestReplicationHandler FAILED

Issue Links

• is related to: SOLR-4161 deadlock if commit+newSearcher occurs during core close, can happen as a result of snappuller (occured in TestReplicationHandler)