[SOLR-14985] Slow indexing and search performance when using HttpClusterStateProvider

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: SolrJ
• Assignee: Unassigned
• Reporter: Shalin Shekhar Mangar
• Created: 05/Nov/20 15:53
• Updated: 13/Feb/21 00:08

Description

HttpClusterStateProvider fetches and caches Aliases and Live Nodes for 5 seconds.

The BaseSolrCloudClient caches DocCollection for 60 seconds but only if the DocCollection is not lazy and all collections returned by HttpClusterStateProvider are not lazy which means they are never cached.

The BaseSolrCloudClient has a method for resolving aliases which fetches DocCollection for each input collection. This is an HTTP call with no caching when using HttpClusterStateProvider. This resolveAliases method is called twice for each update.

So overall, at least 3 HTTP calls are made to fetch cluster state for each update request when using HttpClusterStateProvider. There may be more if aliases are involved or if more than one collection is specified in the request. Similar problems exist on the query path as well.

Due to these reasons, using HttpClusterStateProvider causes horrible latencies and throughput for update and search requests.

Issue Links

• is related to: SOLR-14966 Solr should use cache collection state to validate _stateVer_ sent by SolrJ
• is related to: SOLR-14967 CloudSolrClient does not pass _stateVer_ if ModifiableSolrParams isn't used
• links to: GitHub Pull Request #2086