Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-842

ParallelMultiSearcher memory leak

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Invalid
    • Affects Version/s: 2.1
    • Fix Version/s: 2.1
    • Component/s: core/search
    • Labels:
      None
    • Environment:

      Windows XP SP2 and Red Hat EL 4

    • Lucene Fields:
      New

      Description

      When using the org.apache.lucene.search.ParallelMultiSearcher to search on a single searcher (reading a single index), continuous runs result in a memory leak.
      Substituting the MultiSearcher does not result in a memory leak. and is the workaround currently used.

      And example of the code used is as follows. Note the close routine was added for the individual searchers and the MultiSearcher otherwise the was a leak in MultiSearcher.

      private void doSearch(Search search)
      {
      IndexSearcher[] indexSearchers = null;

      MultiSearcher multiSearcher = null;
      try

      { indexSearchers = getIndexSearcher(); // aggregate the searches across multiple indexes multiSearcher = new ParallelMultiSearcher(indexSearchers); // causes LEAK BAD //multiSearcher = new MultiSearcher(indexSearchers); // NO leak GOOD final QueryParser parser = new QueryParser("content", new ExtendedStandardAnalyser()); final Query query = parser.parse(search.getQuery()); final Hits hits = multiSearcher.search(query, getFilter(search.getFilters()), getSort(search.getSort())); // process hits... }

      finally

      { close(indexSearchers); close(multiSearcher); }

      }

      /**

      • Close the index searchers.
      • @param indexSearchers Index Searchers.
        */
        private static void close(IndexSearcher[] indexSearchers)
        {
        if (indexSearchers != null)
        {
        for (IndexSearcher indexSearcher : indexSearchers)
        Unknown macro: { try { indexSearcher.close(); } catch (IOException ioex) { LOGGER.warn("Unable to close the index searcher!", ioex); } }

        }
        }

      /**

      • Close the multi-searcher.
      • @param aMultiSearcher Index Searchers.
        */
        private static void close(MultiSearcher aMultiSearcher)
        Unknown macro: { try { aMultiSearcher.close(); } catch (IOException ioex) { LOGGER.warn("Unable to close the multi searcher!", ioex); } }

        Attachments

        1. TestParallelMultiSearcherMemLeak.java
          4 kB
          Doron Cohen
        2. search_test_heap.PNG
          105 kB
          Thomas Connolly
        3. search_test_gc.PNG
          113 kB
          Thomas Connolly

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              thomas_connolly Thomas Connolly
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: