Solr
  1. Solr
  2. SOLR-4753

SEVERE: Too many close [count:-1] for SolrCore in logs (4.2.1)

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 4.9, Trunk
    • Component/s: None
    • Labels:
      None

      Description

      a user reported core reference counting issues in 4.2.1...

      http://markmail.org/message/akrrj5o24prasm6e

        Activity

        Mark Miller created issue -
        Hide
        Mark Miller added a comment -

        I'll add an assert to ensure this fails tests.

        Show
        Mark Miller added a comment - I'll add an assert to ensure this fails tests.
        Hide
        Mark Miller added a comment -

        In another issue Yonik Seeley said: When I tried running TestLazyCores, I also saw these errors.

        It seems that's simply because the test calls close too many times.

        Show
        Mark Miller added a comment - In another issue Yonik Seeley said: When I tried running TestLazyCores, I also saw these errors. It seems that's simply because the test calls close too many times.
        Mark Miller made changes -
        Field Original Value New Value
        Description a user reported core reference counting issues:
        http://markmail.org/message/akrrj5o24prasm6e
        Hide
        Commit Tag Bot added a comment -

        [trunk commit] markrmiller
        http://svn.apache.org/viewvc?view=revision&revision=1471114

        SOLR-4753: add assert so that tests fail when SolrCores are closed too many times

        Show
        Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1471114 SOLR-4753 : add assert so that tests fail when SolrCores are closed too many times
        Hide
        Commit Tag Bot added a comment -

        [branch_4x commit] markrmiller
        http://svn.apache.org/viewvc?view=revision&revision=1471116

        SOLR-4753: add assert so that tests fail when SolrCores are closed too many times

        Show
        Commit Tag Bot added a comment - [branch_4x commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1471116 SOLR-4753 : add assert so that tests fail when SolrCores are closed too many times
        Hide
        Hoss Man added a comment -

        details from Yago in thread linked in description...

        I use solr as a SolrCluster, the main feature that I use is faceting to do some analytics and normal queries to do
        free text search and retrieve data using filters.

        I don't use any custom plugin or contribute plugin.

        At the moment I'm importing my data from mysql to solr, I don't use dih, instead I use a custom mechanism. In this
        import, I don't do hard or soft commits, I relay this responsibility to solr.

        I don't know if this info is useful but I have a lot of: WARNING: [XXX] PERFORMANCE WARNING: Overlapping
        onDeckSearchers=2

        The cluster is formed by a thousand of collection, I have a collection for each client.

        <config>
          <luceneMatchVersion>LUCENE_40</luceneMatchVersion>
          <directoryFactory name="DirectoryFactory" class="${solr.directoryFactory:solr.StandardDirectoryFactory}"/>
        
          <indexConfig>
            <ramBufferSizeMB>256</ramBufferSizeMB>
            <mergeFactor>20</mergeFactor>
            <mergeScheduler class="org.apache.lucene.index.ConcurrentMergeScheduler"/>
            <lockType>native</lockType>
          </indexConfig>
        
          <query>
            <enableLazyFieldLoading>true</enableLazyFieldLoading>
            <queryResultWindowSize>1000</queryResultWindowSize>
            <queryResultMaxDocsCached>3000</queryResultMaxDocsCached>
            <maxWarmingSearchers>2</maxWarmingSearchers>
            <useFilterForSortedQuery>true</useFilterForSortedQuery>
            <filterCache
              class="solr.FastLRUCache"
              size="2000"
              initialSize="1500"
              autowarmCount="750"
              cleanupThread="true"/>
            <queryResultCache
              class="solr.FastLRUCache"
              size="2000"
              initialSize="1500"
              autowarmCount="750"
              cleanupThread="true"/>
            <documentCache
              class="solr.FastLRUCache"
              size="20000"
              initialSize="10000"
              autowarmCount="0"
              cleanupThread="true"/>
          </query>
        
          <updateHandler class="solr.DirectUpdateHandler2">
             <updateLog>
                <str name="dir">${solr.data.dir:}</str>
             </updateLog>
             <!-- Commit documents definitions -->
             <autoCommit>
                <maxDocs>5000</maxDocs>
                <maxTime>10000</maxTime>
              </autoCommit>
             <autoSoftCommit>
                <maxTime>2500</maxTime>
             </autoSoftCommit>
             <maxPendingDeletes>20000</maxPendingDeletes>
          </updateHandler>
        
          <requestDispatcher handleSelect="false">
            <requestParsers enableRemoteStreaming="true" multipartUploadLimitInKB="10485760"/>
          </requestDispatcher>
        
          <requestHandler name="/select" class="solr.SearchHandler"/>
          <!--   request handler that returns indented JSON by default -->
          <requestHandler name="/query" class="solr.SearchHandler">
             <lst name="defaults">
               <str name="echoParams">explicit</str>
               <str name="wt">json</str>
               <str name="indent">true</str>
               <str name="df">text</str>
             </lst>
          </requestHandler>
          <!-- realtime get handler, guaranteed to return the latest stored fields of
               any document, without the need to commit or open a new searcher.  The
               current implementation relies on the updateLog feature being enabled. -->
          <requestHandler name="/get" class="solr.RealTimeGetHandler">
             <lst name="defaults">
               <str name="omitHeader">true</str>
               <str name="wt">json</str>
               <str name="indent">false</str>
             </lst>
          </requestHandler>
          <requestHandler name="/admin/" class="solr.admin.AdminHandlers" />
          <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
          <requestHandler name="/update" class="solr.UpdateRequestHandler" />
          <requestHandler name="/analysis/field" startup="lazy" class="solr.solr.FieldAnalysisRequestHandler" />
          <requestHandler name="/analysis/document" class="solr.DocumentAnalysisRequestHandler" startup="lazy" />
          <requestHandler name="/replication" class="solr.ReplicationHandler" startup="lazy" />
          <!-- Echo the request contents back to the client -->
          <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
            <lst name="defaults">
             <str name="echoParams">explicit</str> <!-- for all params (including the default etc) use: 'all' -->
             <str name="echoHandler">true</str>
            </lst>
          </requestHandler>
        
          <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
            <lst name="invariants">
              <str name="q">rows=0;start=0;omitHeader=true</str>
            </lst>
            <lst name="defaults">
              <str name="echoParams">NONE</str>
            </lst>
          </requestHandler>
        
          <!-- config for the admin interface -->
          <admin>
            <defaultQuery>id:*</defaultQuery>
          </admin>
        </config>
        
        

        (NOTE: I pruned some large XML comments from the config posted in email for brevity)

        Show
        Hoss Man added a comment - details from Yago in thread linked in description... I use solr as a SolrCluster, the main feature that I use is faceting to do some analytics and normal queries to do free text search and retrieve data using filters. I don't use any custom plugin or contribute plugin. At the moment I'm importing my data from mysql to solr, I don't use dih, instead I use a custom mechanism. In this import, I don't do hard or soft commits, I relay this responsibility to solr. I don't know if this info is useful but I have a lot of: WARNING: [XXX] PERFORMANCE WARNING: Overlapping onDeckSearchers=2 The cluster is formed by a thousand of collection, I have a collection for each client. <config> <luceneMatchVersion>LUCENE_40</luceneMatchVersion> <directoryFactory name= "DirectoryFactory" class= "${solr.directoryFactory:solr.StandardDirectoryFactory}" /> <indexConfig> <ramBufferSizeMB>256</ramBufferSizeMB> <mergeFactor>20</mergeFactor> <mergeScheduler class= "org.apache.lucene.index.ConcurrentMergeScheduler" /> <lockType> native </lockType> </indexConfig> <query> <enableLazyFieldLoading> true </enableLazyFieldLoading> <queryResultWindowSize>1000</queryResultWindowSize> <queryResultMaxDocsCached>3000</queryResultMaxDocsCached> <maxWarmingSearchers>2</maxWarmingSearchers> <useFilterForSortedQuery> true </useFilterForSortedQuery> <filterCache class= "solr.FastLRUCache" size= "2000" initialSize= "1500" autowarmCount= "750" cleanupThread= " true " /> <queryResultCache class= "solr.FastLRUCache" size= "2000" initialSize= "1500" autowarmCount= "750" cleanupThread= " true " /> <documentCache class= "solr.FastLRUCache" size= "20000" initialSize= "10000" autowarmCount= "0" cleanupThread= " true " /> </query> <updateHandler class= "solr.DirectUpdateHandler2" > <updateLog> <str name= "dir" >${solr.data.dir:}</str> </updateLog> <!-- Commit documents definitions --> <autoCommit> <maxDocs>5000</maxDocs> <maxTime>10000</maxTime> </autoCommit> <autoSoftCommit> <maxTime>2500</maxTime> </autoSoftCommit> <maxPendingDeletes>20000</maxPendingDeletes> </updateHandler> <requestDispatcher handleSelect= " false " > <requestParsers enableRemoteStreaming= " true " multipartUploadLimitInKB= "10485760" /> </requestDispatcher> <requestHandler name= "/select" class= "solr.SearchHandler" /> <!-- request handler that returns indented JSON by default --> <requestHandler name= "/query" class= "solr.SearchHandler" > <lst name= "defaults" > <str name= "echoParams" >explicit</str> <str name= "wt" >json</str> <str name= "indent" > true </str> <str name= "df" >text</str> </lst> </requestHandler> <!-- realtime get handler, guaranteed to return the latest stored fields of any document, without the need to commit or open a new searcher. The current implementation relies on the updateLog feature being enabled. --> <requestHandler name= "/get" class= "solr.RealTimeGetHandler" > <lst name= "defaults" > <str name= "omitHeader" > true </str> <str name= "wt" >json</str> <str name= "indent" > false </str> </lst> </requestHandler> <requestHandler name= "/admin/" class= "solr.admin.AdminHandlers" /> <requestHandler name= "standard" class= "solr.StandardRequestHandler" default = " true " /> <requestHandler name= "/update" class= "solr.UpdateRequestHandler" /> <requestHandler name= "/analysis/field" startup= "lazy" class= "solr.solr.FieldAnalysisRequestHandler" /> <requestHandler name= "/analysis/document" class= "solr.DocumentAnalysisRequestHandler" startup= "lazy" /> <requestHandler name= "/replication" class= "solr.ReplicationHandler" startup= "lazy" /> <!-- Echo the request contents back to the client --> <requestHandler name= "/debug/dump" class= "solr.DumpRequestHandler" > <lst name= "defaults" > <str name= "echoParams" >explicit</str> <!-- for all params (including the default etc) use: 'all' --> <str name= "echoHandler" > true </str> </lst> </requestHandler> <requestHandler name= "/admin/ping" class= "solr.PingRequestHandler" > <lst name= "invariants" > <str name= "q" >rows=0;start=0;omitHeader= true </str> </lst> <lst name= "defaults" > <str name= "echoParams" >NONE</str> </lst> </requestHandler> <!-- config for the admin interface --> <admin> <defaultQuery>id:*</defaultQuery> </admin> </config> (NOTE: I pruned some large XML comments from the config posted in email for brevity)
        Hoss Man made changes -
        Summary SEVERE: Too many close [count:-1] for SolrCore in logs. SEVERE: Too many close [count:-1] for SolrCore in logs (4.2.1)
        Description a user reported core reference counting issues:
        http://markmail.org/message/akrrj5o24prasm6e
        a user reported core reference counting issues in 4.2.1...

        http://markmail.org/message/akrrj5o24prasm6e
        Hide
        Yago Riveiro added a comment -

        Hi,

        Today I upgrade solr to 4.3 and I had the same issue.

        65907 4159431 [node02.solrcloud-startStop-1-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader  – A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
        65908 4159698 [RecoveryThread] INFO  org.apache.solr.cloud.RecoveryStrategy  – Finished recovery process. core=ST-0712
        65909 4159698 [RecoveryThread] INFO  org.apache.solr.core.SolrCore  – [ST-0712]  CLOSING SolrCore org.apache.solr.core.SolrCore@73e004a
        65910 4159698 [RecoveryThread] INFO  org.apache.solr.update.UpdateHandler  – closing DirectUpdateHandler2{commits=0,autocommit maxDocs=5000,autocommit maxTime=10000ms,autocommits=0,soft autocommit maxTime=2500ms,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
        65911 4159699 [RecoveryThread] INFO  org.apache.solr.core.SolrCore  – [ST-0712] Closing main searcher on request.
        65912 4159699 [catalina-exec-12] ERROR org.apache.solr.core.SolrCore  – Too many close [count:-1] on org.apache.solr.core.SolrCore@73e004a. Please report this exception to solr-user@lucene.apache.org
        65913 4159699 [catalina-exec-12] INFO  org.apache.solr.core.CoreContainer  – Persisting cores config to /opt/node02.solrcloud/solr/home/solr.xml
        65914 4160185 [catalina-exec-12] INFO  org.apache.solr.core.SolrXMLSerializer  – Persisting cores config to /opt/node02.solrcloud/solr/home/solr.xml
        
        Show
        Yago Riveiro added a comment - Hi, Today I upgrade solr to 4.3 and I had the same issue. 65907 4159431 [node02.solrcloud-startStop-1-EventThread] INFO org.apache.solr.common.cloud.ZkStateReader – A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4) 65908 4159698 [RecoveryThread] INFO org.apache.solr.cloud.RecoveryStrategy – Finished recovery process. core=ST-0712 65909 4159698 [RecoveryThread] INFO org.apache.solr.core.SolrCore – [ST-0712] CLOSING SolrCore org.apache.solr.core.SolrCore@73e004a 65910 4159698 [RecoveryThread] INFO org.apache.solr.update.UpdateHandler – closing DirectUpdateHandler2{commits=0,autocommit maxDocs=5000,autocommit maxTime=10000ms,autocommits=0,soft autocommit maxTime=2500ms,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0} 65911 4159699 [RecoveryThread] INFO org.apache.solr.core.SolrCore – [ST-0712] Closing main searcher on request. 65912 4159699 [catalina-exec-12] ERROR org.apache.solr.core.SolrCore – Too many close [count:-1] on org.apache.solr.core.SolrCore@73e004a. Please report this exception to solr-user@lucene.apache.org 65913 4159699 [catalina-exec-12] INFO org.apache.solr.core.CoreContainer – Persisting cores config to /opt/node02.solrcloud/solr/home/solr.xml 65914 4160185 [catalina-exec-12] INFO org.apache.solr.core.SolrXMLSerializer – Persisting cores config to /opt/node02.solrcloud/solr/home/solr.xml
        Hide
        Steve Rowe added a comment -

        Bulk move 4.4 issues to 4.5 and 5.0

        Show
        Steve Rowe added a comment - Bulk move 4.4 issues to 4.5 and 5.0
        Steve Rowe made changes -
        Fix Version/s 4.5 [ 12324743 ]
        Fix Version/s 4.4 [ 12324324 ]
        Adrien Grand made changes -
        Fix Version/s 4.6 [ 12325000 ]
        Fix Version/s 5.0 [ 12321664 ]
        Fix Version/s 4.5 [ 12324743 ]
        Uwe Schindler made changes -
        Fix Version/s 4.7 [ 12325573 ]
        Fix Version/s 4.6 [ 12325000 ]
        David Smiley made changes -
        Fix Version/s 4.8 [ 12326254 ]
        Fix Version/s 4.7 [ 12325573 ]
        Hide
        Uwe Schindler added a comment -

        Move issue to Solr 4.9.

        Show
        Uwe Schindler added a comment - Move issue to Solr 4.9.
        Uwe Schindler made changes -
        Fix Version/s 4.9 [ 12326731 ]
        Fix Version/s 5.0 [ 12321664 ]
        Fix Version/s 4.8 [ 12326254 ]

          People

          • Assignee:
            Mark Miller
            Reporter:
            Mark Miller
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development