[SOLR-13365] Add autoscaling policy support for balancing cores across all availability zones

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Fix Version/s: 8.2, 9.0
• Component/s: AutoScaling
• Assignee: Shalin Shekhar Mangar
• Reporter: Shalin Shekhar Mangar
• Created: 03/Apr/19 13:46
• Updated: 12/May/22 00:26
• Resolved: 14/Aug/21 01:33

Description

The autoscaling policy supports spreading replicas of a shard across zones (via sysprops) but that by itself does not guarantee that all availability zones are actually used and that too in a balanced way.

For example:

{replica:#EQUAL, shard:#EACH, sysprop.az:#EACH}

The above policy might end up using only 2 out of 3 availability zones to spread the replicas (assuming there are enough nodes per zone to not violate any other policy rules) and not generate any violations. So although we still have resilience against the loss of one AZ, we do not keep enough capacity as we could have if we had used all 3 zones equally. In summary, this is a cluster balancing problem.