Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-14985

Slow indexing and search performance when using HttpClusterStateProvider

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: SolrJ
    • Labels:
      None

      Description

      HttpClusterStateProvider fetches and caches Aliases and Live Nodes for 5 seconds.

      The BaseSolrCloudClient caches DocCollection for 60 seconds but only if the DocCollection is not lazy and all collections returned by HttpClusterStateProvider are not lazy which means they are never cached.

      The BaseSolrCloudClient has a method for resolving aliases which fetches DocCollection for each input collection. This is an HTTP call with no caching when using HttpClusterStateProvider. This resolveAliases method is called twice for each update.

      So overall, at least 3 HTTP calls are made to fetch cluster state for each update request when using HttpClusterStateProvider. There may be more if aliases are involved or if more than one collection is specified in the request. Similar problems exist on the query path as well.

      Due to these reasons, using HttpClusterStateProvider causes horrible latencies and throughput for update and search requests.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                shalin Shalin Shekhar Mangar
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h