[SOLR-10397] Port 'autoAddReplicas' feature to the autoscaling framework and make it work with non-shared filesystems

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 7.1, 8.0
• Component/s: SolrCloud
• Labels: autoscaling
• Assignee: Cao Manh Dat
• Reporter: Shalin Shekhar Mangar
• Created: 31/Mar/17 14:38
• Updated: 18/Mar/20 02:39
• Resolved: 17/Oct/17 10:47

Description

Currently 'autoAddReplicas=true' can be specified in the Collection Create API to automatically add replicas when a replica becomes unavailable. I propose to move this feature to the autoscaling cluster policy rules design.

This will include the following:

• Trigger support for ‘nodeLost’ event type
• Modification of existing implementation of ‘autoAddReplicas’ to automatically create the appropriate ‘nodeLost’ trigger.
• Any such auto-created trigger must be marked internally such that setting ‘autoAddReplicas=false’ via the Modify Collection API should delete or disable corresponding trigger.
• Support for non-HDFS filesystems while retaining the optimization afforded by HDFS i.e. the replaced replica can point to the existing data dir of the old replica.
• Deprecate/remove the feature of enabling/disabling ‘autoAddReplicas’ across the entire cluster using cluster properties in favor of using the suspend-trigger/resume-trigger APIs.

This will retain backward compatibility for the most part and keep a common use-case easy to enable as well as make it available to more people (i.e. people who don't use HDFS).

Issue Links

• is blocked by: SOLR-11045 The new replica created by MoveReplica will have to have same name and coreName as the old one in case of HDFS
• is blocked by: SOLR-11011 Assign.buildCoreName can lead to error in creating a new core when legacyCloud=false
• relates to: SOLR-11591 AutoAddReplicasIntegrationTest failures on jenkins
• relates to: SOLR-12114 Changes made to 'autoReplicaFailoverWaitAfterExpiration' in solr.xml is not reflected automatically in the autoAddReplicas autoscaling trigger