Lucene - Core
  1. Lucene - Core
  2. LUCENE-3409

NRT reader/writer over RAMDirectory memory leak

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.2, 3.3, 4.0-ALPHA
    • Fix Version/s: 3.4, 4.0-ALPHA
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      with NRT reader/writer, emptying an index using:
      writer.deleteAll()
      writer.commit()
      doesn't release all allocated memory.

      for example the following code will generate a memory leak:

      /**

      • Reveals a memory leak in NRT reader/writer<br>
      • The following main() does 10K cycles of:
      • <ul>
      • <li>Add 10K empty documents to index writer</li>
      • <li>commit()</li>
      • <li>open NRT reader over the writer, and immediately close it</li>
      • <li>delete all documents from the writer</li>
      • <li>commit changes to the writer</li>
      • </ul>
      • Running with -Xmx256M results in an OOME after ~2600 cycles
        */
        public static void main(String[] args) throws Exception {
        RAMDirectory d = new RAMDirectory();
        IndexWriter w = new IndexWriter(d, new IndexWriterConfig(Version.LUCENE_33, new KeywordAnalyzer()));
        Document doc = new Document();

      for(int i = 0; i < 10000; i++) {
      for(int j = 0; j < 10000; ++j)

      { w.addDocument(doc); }

      w.commit();
      IndexReader.open(w, true).close();

      w.deleteAll();
      w.commit();
      }

      w.close();
      d.close();
      }

        Activity

        Hide
        Gilad Barkai added a comment -

        This issue is relevant for trunk as well.
        Please update the Affected versions accordingly.

        Show
        Gilad Barkai added a comment - This issue is relevant for trunk as well. Please update the Affected versions accordingly.
        Hide
        Michael McCandless added a comment -

        I found the issue: we are failing to drop pool'd readers in IW.deleteAll. I'll commit fix shortly.

        Show
        Michael McCandless added a comment - I found the issue: we are failing to drop pool'd readers in IW.deleteAll. I'll commit fix shortly.
        Hide
        Michael McCandless added a comment -

        Thanks tal!

        Show
        Michael McCandless added a comment - Thanks tal!

          People

          • Assignee:
            Michael McCandless
            Reporter:
            tal steier
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development