Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-6146

Leak in CloudSolrServer causing "Too many open files"

    Details

      Description

      Due to a misconfiguration in one of our QA clusters, we uncovered a leak in CloudSolrServer. If this line throws:

      https://github.com/apache/lucene-solr/blob/branch_4x/solr/solrj/src/java/org/apache/solr/client/solrj/impl/CloudSolrServer.java#L242

      then the instantiated ZkStateReader is leaked.

      Here's the stacktrace of the Exception (we're using a custom build so the line numbers won't quite match up, but it gives the idea):
      at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:304) at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:568) at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:557) at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117) at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:33) at com.apple.cie.search.client.crossdc.MirroredSolrRequestHandler.handleItem(MirroredSolrRequestHandler.java:100) at com.apple.cie.search.client.crossdc.MirroredSolrRequestHandler.handleItem(MirroredSolrRequestHandler.java:33) at com.apple.coda.queueing.CodaQueueConsumer$StreamProcessor.run(CodaQueueConsumer.java:147) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /live_nodes at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468) at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256) at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73) at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253) at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:305) at org.apache.solr.client.solrj.impl.CloudSolrServer.createZkStateReader(CloudSolrServer.java:935) at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:298) ... 10 more

        Attachments

        1. SOLR-6146.patch
          3 kB
          Shalin Shekhar Mangar
        2. SOLR-6146.patch
          3 kB
          Shalin Shekhar Mangar
        3. SOLR-6146.patch
          1 kB
          Varun Thacker
        4. SOLR-6146.patch
          2 kB
          Varun Thacker

          Activity

            People

            • Assignee:
              shalinmangar Shalin Shekhar Mangar
              Reporter:
              mewmewball Jessica Cheng Mallet
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: