[SOLR-15014] Runaway replica creation with autoscaling example from ref guide

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 8.6.3
• Component/s: AutoScaling
• Assignee: Unassigned
• Reporter: Gus Heck
• Created: 23/Nov/20 16:44
• Updated: 14/Aug/21 01:33
• Resolved: 14/Aug/21 01:33

Description

Although the present autoscaling implementation is deprecated, I have a client intent on using it, and in trying to create rules that ensure all replicas on all nodes, I wound up getting into a state where one replica was (apparently) infinitely creating new copies of itself. The boiled down steps to reproduce:

Create a 4 node cluster locally for testing from a checkout of the tagged version for 8.6.3

(Using solr/cloud-dev/cloud.sh)

./cloud.sh  new -r   

Create a collection

http://localhost:8983/solr/admin/collections?action=CREATE&name=newCollection&numShards=2&replicationFactor=1

Add this trigger from the ref guide (https://lucene.apache.org/solr/guide/8_6/solrcloud-autoscaling-triggers.html#node-added-trigger):

{
  "set-trigger": {
    "name": "node_added_trigger",
    "event": "nodeAdded",
    "waitFor": "5s",
    "preferredOperation": "ADDREPLICA",
    "replicaType": "PULL"
  }
}

Reboot the cluster, and when it comes up infinite replica creation ensues (attaching screen shot of admin UI showing replicated shard momentarily)