Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-13790

LRUStatsCache size explosion and ineffective caching

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 7.7.2, 8.2, 8.3
    • Fix Version/s: 8.3
    • Component/s: None
    • Labels:
      None

      Description

      On a sizeable cluster with multi-shard multi-replica collections, when LRUStatsCache was in use we encountered excessive memory usage, which consequently led to severe performance problems.

      On a closer examination of the heapdumps it became apparent that when LRUStatsCache.addToPerShardTermStats is called it creates instances of FastLRUCache using the passed shard argument - however, the value of this argument is not a simple shard name but instead it's a randomly ordered list of ALL replica URLs for this shard.

      As a result, due to the combinatoric number of possible keys, over time the map in LRUStatsCache.perShardTemStats grew to contain ~2 mln entries...

      The fix seems to be simply to extract the shard name and cache using this name instead of the full string value of the shard parameter. Existing unit tests also need much improvement.

        Attachments

        1. SOLR-13790.patch
          50 kB
          Andrzej Bialecki
        2. SOLR-13790.patch
          82 kB
          Andrzej Bialecki
        3. SOLR-13790.patch
          101 kB
          Andrzej Bialecki

          Activity

            People

            • Assignee:
              ab Andrzej Bialecki
              Reporter:
              ab Andrzej Bialecki
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: