[SOLR-15019] Replica placement API needs a way to fetch existing replica metrics

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 9.0
• Assignee: Andrzej Bialecki
• Reporter: Andrzej Bialecki
• Created: 26/Nov/20 09:52
• Updated: 18/May/23 15:13
• Resolved: 26/Apr/21 15:51

Description

Replica placement API was introduced in SOLR-14613. It offers a few sample (and simple) implementations of placement plugins.

However, this API doesn't offer support for retrieving per-replica metrics, which are required for calculating more realistic placements. For example, when calculating placements for ADDREPLICA on an already existing collection the plugin should know what is the size of replica in order to avoid placing large replicas on nodes with insufficient free disk space.

After discussing this with ilan we propose the following additions to the API:

• use the existing AttributeFetcher interface as a facade for retrieving per-replica values (currently it only retrieves per-node values)
• add ShardValues interface to represent strongly-typed API for key metrics, such as replica size, number of docs, number of update and search requests.

Plugins could then use this API like this:

AttributeFetcher attributeFetcher = ...
SolrCollection solrCollection = ...
Set<String> metricNames = ...
attributeFetcher.requestCollectionMetrics(solrCollection, solrCollection.getShardNames(), metricNames);

AttributeValues attributeValues = attributeFetcher.fetchAttributes();
ShardValues shardValues = attributeValues.getShardMetrics(solrCollection.getName(), shardName);
int sizeInGB = shardValues.getSizeInGB(); // retrieves shard leader metrics
int replicaSizeInGB = shardValues.getSizeInGB(replica);

Issue Links

• causes: SOLR-16808 Solr publishes environment variables via the Metrics API
• links to: GitHub Pull Request #2133