Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-14172

Collection metadata remains in zookeeper if too many shards requested

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 8.3.1
    • Fix Version/s: master (9.0), 8.5
    • Component/s: None
    • Labels:
      None

      Description

      When I try to create a collection and request too many shards, collection creation fails with the expected error message:

      $ curl -i --retry 5 -s -L -k --negotiate -u : 'http://asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:8983/solr/admin/collections?action=CREATE&name=TooManyShardstest1&numShards=4&collection.configName=zk_init_too&maxShardsPerNode=1'
      HTTP/1.1 400 Bad Request
      Content-Type: application/json;charset=utf-8
      Content-Length: 1562
      
      
      {
        "responseHeader":{
          "status":400,
          "QTime":122},
        "Operation create caused exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)",
        "exception":{
          "msg":"Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)",
          "rspCode":400},
        "error":{
          "metadata":[
            "error-class","org.apache.solr.common.SolrException",
            "root-error-class","org.apache.solr.common.SolrException"],
          "msg":"Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)",
          "code":400}}
      

      Although the collection creation was not successful if I list the collections it shows the new collection:

      $ solr  collection --list                                        
      TooManyShardstest1 (1) 
       

      Looks like metadata remains in Zookeeper:

      $ zkcli.sh -zkhost asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:2181/solr -cmd ls /collections
      INFO  - 2020-01-06 04:54:01.851; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
      INFO  - 2020-01-06 04:54:01.880; org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
      INFO  - 2020-01-06 04:54:01.881; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
      /collections (1)
       /collections/TooManyShardstest1 (1)
       DATA:
           {"configName":"zk_init_too"}
        /collections/TooManyShardstest1/state.json (0)
        DATA:
            {"TooManyShardstest1":{
                "pullReplicas":"0",
                "replicationFactor":"1",
                "router":{"name":"compositeId"},
                "maxShardsPerNode":"1",
                "autoAddReplicas":"false",
                "nrtReplicas":"1",
                "tlogReplicas":"0",
                "shards":{
                  "shard1":{
                    "range":"80000000-bfffffff",
                    "state":"active",
                    "replicas":{}},
                  "shard2":{
                    "range":"c0000000-ffffffff",
                    "state":"active",
                    "replicas":{}},
                  "shard3":{
                    "range":"0-3fffffff",
                    "state":"active",
                    "replicas":{}},
                  "shard4":{
                    "range":"40000000-7fffffff",
                    "state":"active",
                    "replicas":{}}}}}
      

       

        Attachments

        1. SOLR-14172.patch
          4 kB
          Shalin Shekhar Mangar
        2. SOLR-14172.patch
          4 kB
          Shalin Shekhar Mangar

          Issue Links

            Activity

              People

              • Assignee:
                shalin Shalin Shekhar Mangar
                Reporter:
                asalamon74 Andras Salamon
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m