Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7062

CLUSTERSTATUS returns a collection with state=active, even though the collection could not be created due to a missing configSet

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Cannot Reproduce
    • 4.10.3
    • None
    • SolrCloud

    Description

      A collection can not be created, if its configSet does not exist. Nevertheless, a subsequent CLUSTERSTATUS CollectionAdminRequest returns this collection with a state=active.

      See log below.

      [INFO] Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000110 message:{
        "operation":"createcollection",
        "fromApi":"true",
        "name":"blueprint_media_comments",
        "collection.configName":"elastic",
        "numShards":"1",
        "property.dataDir":"data",
        "property.instanceDir":"cores/blueprint_media_comments"}
      [WARNING] OverseerCollectionProcessor.processMessage : createcollection , {
        "operation":"createcollection",
        "fromApi":"true",
        "name":"blueprint_media_comments",
        "collection.configName":"elastic",
        "numShards":"1",
        "property.dataDir":"data",
        "property.instanceDir":"cores/blueprint_media_comments"}
      [INFO] creating collections conf node /collections/blueprint_media_comments 
      [INFO] makePath: /collections/blueprint_media_comments
      [INFO] Got user-level KeeperException when processing sessionid:0x14b315b0f4a000e type:create cxid:0x2f2e zxid:0x2f4 txntype:-1 reqpath:n/a Error Path:/overseer Error:KeeperErrorCode = NodeExists for /overseer
      [INFO] LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
      [INFO] building a new collection: blueprint_media_comments
      [INFO] Create collection blueprint_media_comments with shards [shard1]
      [INFO] A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
      [INFO] Creating SolrCores for new collection blueprint_media_comments, shardNames [shard1] , replicationFactor : 1
      [INFO] Creating shard blueprint_media_comments_shard1_replica1 as part of slice shard1 of collection blueprint_media_comments on localhost:44080_solr
      [INFO] core create command qt=/admin/cores&property.dataDir=data&collection.configName=elastic&name=blueprint_media_comments_shard1_replica1&action=CREATE&numShards=1&collection=blueprint_media_comments&shard=shard1&wt=javabin&version=2&property.instanceDir=cores/blueprint_media_comments
      [INFO] publishing core=blueprint_media_comments_shard1_replica1 state=down collection=blueprint_media_comments
      [INFO] LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
      [INFO] look for our core node name
      [INFO] Update state numShards=1 message={
        "core":"blueprint_media_comments_shard1_replica1",
        "roles":null,
        "base_url":"http://localhost:44080/solr",
        "node_name":"localhost:44080_solr",
        "numShards":"1",
        "state":"down",
        "shard":"shard1",
        "collection":"blueprint_media_comments",
        "operation":"state"}
      [INFO] A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
      [INFO] waiting to find shard id in clusterstate for blueprint_media_comments_shard1_replica1
      [INFO] Check for collection zkNode:blueprint_media_comments
      [INFO] Collection zkNode exists
      [INFO] Load collection config from:/collections/blueprint_media_comments
      [ERROR] Specified config does not exist in ZooKeeper:elastic
      [ERROR] Error creating core [blueprint_media_comments_shard1_replica1]: Specified config does not exist in ZooKeeper:elastic
      org.apache.solr.common.cloud.ZooKeeperException: Specified config does not exist in ZooKeeper:elastic
      	at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:160)
      	at org.apache.solr.cloud.CloudConfigSetService.createCoreResourceLoader(CloudConfigSetService.java:37)
      	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:58)
      	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:489)
      	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:466)
      	at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:575)
      	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:199)
      	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:188)
      	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
      	at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:729)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:258)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
      	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
      	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
      	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
      	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
      	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
      	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
      	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
      	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
      	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      	at java.lang.Thread.run(Thread.java:745)
      [ERROR] org.apache.solr.common.SolrException: Error CREATEing SolrCore 'blueprint_media_comments_shard1_replica1': Unable to create core [blueprint_media_comments_shard1_replica1] Caused by: Specified config does not exist in ZooKeeper:elastic
      	at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:613)
      	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:199)
      	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:188)
      	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
      	at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:729)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:258)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
      	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
      	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
      	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
      	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
      	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
      	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
      	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
      	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
      	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.solr.common.SolrException: Unable to create core [blueprint_media_comments_shard1_replica1]
      	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:507)
      	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:466)
      	at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:575)
      	... 22 more
      Caused by: org.apache.solr.common.cloud.ZooKeeperException: Specified config does not exist in ZooKeeper:elastic
      	at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:160)
      	at org.apache.solr.cloud.CloudConfigSetService.createCoreResourceLoader(CloudConfigSetService.java:37)
      	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:58)
      	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:489)
      	... 24 more
      
      [INFO] [admin] webapp=null path=/admin/cores params={qt=/admin/cores&property.dataDir=data&collection.configName=elastic&name=blueprint_media_comments_shard1_replica1&action=CREATE&numShards=1&collection=blueprint_media_comments&shard=shard1&wt=javabin&version=2&property.instanceDir=cores/blueprint_media_comments} status=400 QTime=1005 
      [ERROR] Error from shard: http://localhost:44080/solr
      org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error CREATEing SolrCore 'blueprint_media_comments_shard1_replica1': Unable to create core [blueprint_media_comments_shard1_replica1] Caused by: Specified config does not exist in ZooKeeper:elastic
      	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
      	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
      	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
      	at org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:157)
      	at org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:119)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      [INFO] Finished create command on all shards for collection: blueprint_media_comments
      [INFO] LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
      [INFO] Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000110 complete, response:{failure={null=org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error CREATEing SolrCore 'blueprint_media_comments_shard1_replica1': Unable to create core [blueprint_media_comments_shard1_replica1] Caused by: Specified config does not exist in ZooKeeper:elastic}}
      [INFO] LatchChildWatcher fired on path: /overseer/collection-queue-work/qnr-0000000110 state: SyncConnected type NodeDataChanged
      [INFO] [admin] webapp=null path=/admin/collections params={property.dataDir=data&collection.configName=elastic&name=blueprint_media_comments&action=CREATE&numShards=1&wt=javabin&version=2&property.instanceDir=cores/blueprint_media_comments} status=0 QTime=1134 
      [INFO] LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
      [INFO] LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
      [WARNING] OverseerCollectionProcessor.processMessage : clusterstatus , {"operation":"clusterstatus"}
      [INFO] Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000112 complete, response:{cluster={collections={live={replicationFactor=1, shards={shard1={range=80000000-7fffffff, state=active, replicas={live={core=live, base_url=http://localhost:44080/solr, node_name=localhost:44080_solr, state=active, leader=true}}}}, router={name=compositeId}, maxShardsPerNode=1, autoAddReplicas=false, autoCreated=true},studio={replicationFactor=1, shards={shard1={range=80000000-7fffffff, state=active, replicas={studio={core=studio, base_url=http://localhost:44080/solr, node_name=localhost:44080_solr, state=active, leader=true}}}}, router={name=compositeId}, maxShardsPerNode=1, autoAddReplicas=false, autoCreated=true},preview={replicationFactor=1, shards={shard1={range=80000000-7fffffff, state=active, replicas={preview={core=preview, base_url=http://localhost:44080/solr, node_name=localhost:44080_solr, state=active, leader=true}}}}, router={name=compositeId}, maxShardsPerNode=1, autoAddReplicas=false, autoCreated=true},blueprint_media_comments={replicationFactor=1, shards={shard1={range=80000000-7fffffff, state=active, replicas={core_node1={core=blueprint_media_comments_shard1_replica1, base_url=http://localhost:44080/solr, node_name=localhost:44080_solr, state=down}}}}, router={name=compositeId}, maxShardsPerNode=1, autoAddReplicas=false}},live_nodes=[localhost:44080_solr]}}
      

      Attachments

        1. SOLR-7062.patch
          4 kB
          Vitaliy Zhovtyuk

        Issue Links

          Activity

            People

              noble.paul Noble Paul
              ng_agi Ng Agi
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: