Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10104

BlockDirectoryCache release hooks do not work with multiple directories

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.4
    • Fix Version/s: 6.5, 7.0
    • Component/s: hdfs
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      https://github.com/apache/lucene-solr/blob/5738c293f0c3f346b3e3e52c937183060d59cba1/solr/core/src/java/org/apache/solr/store/blockcache/BlockDirectoryCache.java#L53

          if (releaseBlocks) {
            keysToRelease = Collections.newSetFromMap(new ConcurrentHashMap<BlockCacheKey,Boolean>(1024, 0.75f, 512));
            blockCache.setOnRelease(new OnRelease() {
              
              @Override
              public void release(BlockCacheKey key) {
                keysToRelease.remove(key);
              }
            });
          }
      

      If we're using the global block cache option and create multiple directories using the same factory, we will lose the release hook for the first directory. I think we can verify that by creating a server with multiple cores.

        Issue Links

          Activity

          Hide
          markrmiller@gmail.com Mark Miller added a comment -

          Great catch!

          Show
          markrmiller@gmail.com Mark Miller added a comment - Great catch!
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit a1f114f70f3800292c25be08213edf39b3e37f6a in lucene-solr's branch refs/heads/master from markrmiller
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a1f114f ]

          SOLR-10104: BlockDirectoryCache release hooks do not work with multiple directories.

          1. Conflicts:
          2. solr/CHANGES.txt
          Show
          jira-bot ASF subversion and git services added a comment - Commit a1f114f70f3800292c25be08213edf39b3e37f6a in lucene-solr's branch refs/heads/master from markrmiller [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a1f114f ] SOLR-10104 : BlockDirectoryCache release hooks do not work with multiple directories. Conflicts: solr/CHANGES.txt
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 7f77f71201250c143c0438d5bb34a59ee964d41b in lucene-solr's branch refs/heads/branch_6x from markrmiller
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7f77f71 ]

          SOLR-10104: BlockDirectoryCache release hooks do not work with multiple directories.

          Show
          jira-bot ASF subversion and git services added a comment - Commit 7f77f71201250c143c0438d5bb34a59ee964d41b in lucene-solr's branch refs/heads/branch_6x from markrmiller [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7f77f71 ] SOLR-10104 : BlockDirectoryCache release hooks do not work with multiple directories.
          Hide
          markrmiller@gmail.com Mark Miller added a comment -

          Thanks Mike!

          Show
          markrmiller@gmail.com Mark Miller added a comment - Thanks Mike!
          Hide
          yseeley@gmail.com Yonik Seeley added a comment -

          Related, doesn't this now mean that removal listeners (set by setOnRelease) will grow without bound for a non-global BlockCache?
          Is it important to continue supporting a non-global BlockCache?

          Show
          yseeley@gmail.com Yonik Seeley added a comment - Related, doesn't this now mean that removal listeners (set by setOnRelease) will grow without bound for a non-global BlockCache? Is it important to continue supporting a non-global BlockCache?
          Hide
          markrmiller@gmail.com Mark Miller added a comment -

          I have been considering the non global cache a bug. Just kept it around for
          Back compact mostly. And that global is kind of configured funky being in solrconfig.xml.

          Show
          markrmiller@gmail.com Mark Miller added a comment - I have been considering the non global cache a bug. Just kept it around for Back compact mostly. And that global is kind of configured funky being in solrconfig.xml.
          Hide
          mdrob Mike Drob added a comment -

          I have been considering the non global cache a bug. Just kept it around for
          Back compact mostly. And that global is kind of configured funky being in solrconfig.xml.

          I filed SOLR-10222 for this

          Show
          mdrob Mike Drob added a comment - I have been considering the non global cache a bug. Just kept it around for Back compact mostly. And that global is kind of configured funky being in solrconfig.xml. I filed SOLR-10222 for this

            People

            • Assignee:
              markrmiller@gmail.com Mark Miller
              Reporter:
              mdrob Mike Drob
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development