[SOLR-14172] Collection metadata remains in zookeeper if too many shards requested

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 8.3.1
• Fix Version/s: 8.5, 9.0
• Assignee: Shalin Shekhar Mangar
• Reporter: Andras Salamon
• Created: 07/Jan/20 12:26
• Updated: 24/Mar/20 12:45
• Resolved: 23/Jan/20 09:56

Description

When I try to create a collection and request too many shards, collection creation fails with the expected error message:

$ curl -i --retry 5 -s -L -k --negotiate -u : 'http://asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:8983/solr/admin/collections?action=CREATE&name=TooManyShardstest1&numShards=4&collection.configName=zk_init_too&maxShardsPerNode=1'
HTTP/1.1 400 Bad Request
Content-Type: application/json;charset=utf-8
Content-Length: 1562


{
  "responseHeader":{
    "status":400,
    "QTime":122},
  "Operation create caused exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)",
  "exception":{
    "msg":"Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)",
    "rspCode":400},
  "error":{
    "metadata":[
      "error-class","org.apache.solr.common.SolrException",
      "root-error-class","org.apache.solr.common.SolrException"],
    "msg":"Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)",
    "code":400}}

Although the collection creation was not successful if I list the collections it shows the new collection:

$ solr  collection --list                                        
TooManyShardstest1 (1) 
 

Looks like metadata remains in Zookeeper:

$ zkcli.sh -zkhost asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:2181/solr -cmd ls /collections
INFO  - 2020-01-06 04:54:01.851; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
INFO  - 2020-01-06 04:54:01.880; org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
INFO  - 2020-01-06 04:54:01.881; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
/collections (1)
 /collections/TooManyShardstest1 (1)
 DATA:
     {"configName":"zk_init_too"}
  /collections/TooManyShardstest1/state.json (0)
  DATA:
      {"TooManyShardstest1":{
          "pullReplicas":"0",
          "replicationFactor":"1",
          "router":{"name":"compositeId"},
          "maxShardsPerNode":"1",
          "autoAddReplicas":"false",
          "nrtReplicas":"1",
          "tlogReplicas":"0",
          "shards":{
            "shard1":{
              "range":"80000000-bfffffff",
              "state":"active",
              "replicas":{}},
            "shard2":{
              "range":"c0000000-ffffffff",
              "state":"active",
              "replicas":{}},
            "shard3":{
              "range":"0-3fffffff",
              "state":"active",
              "replicas":{}},
            "shard4":{
              "range":"40000000-7fffffff",
              "state":"active",
              "replicas":{}}}}}

 

Issue Links

• links to: GitHub Pull Request #1152