Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7657

Queries that reference a TermContext can cause a memory leak when they are cached

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.0, 5.5.4, 6.5, 6.4.1
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      The TermContext class has a reference to the top reader context of the IndexReader that was used to build it. So if you build a (Span)TermQuery that references a TermContext and this query gets cached, then it will keep holding a reference to the index reader, even after the latter gets closed.

      1. LUCENE-7657.patch
        10 kB
        Adrien Grand
      2. LUCENE-7657.patch
        3 kB
        Adrien Grand

        Issue Links

          Activity

          Hide
          jpountz Adrien Grand added a comment -

          Here is a patch that makes sure the queries returned by Weight.getQuery() do not reference a TermContext. An alternative way to fix the issue would be to remove the reference from TermContext on the reader context, but it is not straightforward. Opinions welcome.

          Show
          jpountz Adrien Grand added a comment - Here is a patch that makes sure the queries returned by Weight.getQuery() do not reference a TermContext . An alternative way to fix the issue would be to remove the reference from TermContext on the reader context, but it is not straightforward. Opinions welcome.
          Hide
          jpountz Adrien Grand added a comment -

          I was doing more testing and this patch does not work. We only unset the context in createWeight, so wrapper queries like BooleanQuery or ToParentBlockJoinQuery would still have a reference to the index reader if the wrap a TermQuery that has a TermContext. So I guess the only way would be to remove the reference from TermContext to the index readers.

          Show
          jpountz Adrien Grand added a comment - I was doing more testing and this patch does not work. We only unset the context in createWeight, so wrapper queries like BooleanQuery or ToParentBlockJoinQuery would still have a reference to the index reader if the wrap a TermQuery that has a TermContext . So I guess the only way would be to remove the reference from TermContext to the index readers.
          Hide
          jpountz Adrien Grand added a comment -

          Here is a patch that removes the reader context reference from TermContext.

          Show
          jpountz Adrien Grand added a comment - Here is a patch that removes the reader context reference from TermContext .
          Hide
          mikemccand Michael McCandless added a comment -

          +1

          Clever solution to factor out this Object identity; I wonder if we should fix IndexReader.getXXXCacheKey() similarly ... later!

          Show
          mikemccand Michael McCandless added a comment - +1 Clever solution to factor out this Object identity ; I wonder if we should fix IndexReader.getXXXCacheKey() similarly ... later!
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit f5301428452ee5f9145ef4ecb889442d4e09f1cb in lucene-solr's branch refs/heads/master from Adrien Grand
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f530142 ]

          LUCENE-7657: Fixed potential memory leak when a (Span)TermQuery that wraps a TermContext is cached.

          Show
          jira-bot ASF subversion and git services added a comment - Commit f5301428452ee5f9145ef4ecb889442d4e09f1cb in lucene-solr's branch refs/heads/master from Adrien Grand [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f530142 ] LUCENE-7657 : Fixed potential memory leak when a (Span)TermQuery that wraps a TermContext is cached.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 271dc1ce41129ef2427b3886e7c5c9c0c8830771 in lucene-solr's branch refs/heads/branch_6x from Adrien Grand
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=271dc1c ]

          LUCENE-7657: Fixed potential memory leak when a (Span)TermQuery that wraps a TermContext is cached.

          Show
          jira-bot ASF subversion and git services added a comment - Commit 271dc1ce41129ef2427b3886e7c5c9c0c8830771 in lucene-solr's branch refs/heads/branch_6x from Adrien Grand [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=271dc1c ] LUCENE-7657 : Fixed potential memory leak when a (Span)TermQuery that wraps a TermContext is cached.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit bfb8a6124ade651a308418cc8320abd3e7cdb398 in lucene-solr's branch refs/heads/branch_6_4 from Adrien Grand
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bfb8a61 ]

          LUCENE-7657: Fixed potential memory leak when a (Span)TermQuery that wraps a TermContext is cached.

          Show
          jira-bot ASF subversion and git services added a comment - Commit bfb8a6124ade651a308418cc8320abd3e7cdb398 in lucene-solr's branch refs/heads/branch_6_4 from Adrien Grand [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bfb8a61 ] LUCENE-7657 : Fixed potential memory leak when a (Span)TermQuery that wraps a TermContext is cached.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit f5301428452ee5f9145ef4ecb889442d4e09f1cb in lucene-solr's branch refs/heads/apiv2 from Adrien Grand
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f530142 ]

          LUCENE-7657: Fixed potential memory leak when a (Span)TermQuery that wraps a TermContext is cached.

          Show
          jira-bot ASF subversion and git services added a comment - Commit f5301428452ee5f9145ef4ecb889442d4e09f1cb in lucene-solr's branch refs/heads/apiv2 from Adrien Grand [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f530142 ] LUCENE-7657 : Fixed potential memory leak when a (Span)TermQuery that wraps a TermContext is cached.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit af77c60a71f3d26848641104e4e3f44d23e4e4a9 in lucene-solr's branch refs/heads/branch_5_5 from Adrien Grand
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=af77c60 ]

          LUCENE-7657: Fixed potential memory leak when a (Span)TermQuery that wraps a TermContext is cached.

          Show
          jira-bot ASF subversion and git services added a comment - Commit af77c60a71f3d26848641104e4e3f44d23e4e4a9 in lucene-solr's branch refs/heads/branch_5_5 from Adrien Grand [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=af77c60 ] LUCENE-7657 : Fixed potential memory leak when a (Span)TermQuery that wraps a TermContext is cached.

            People

            • Assignee:
              Unassigned
              Reporter:
              jpountz Adrien Grand
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development