Lucene.Net
  1. Lucene.Net
  2. LUCENENET-488

Can't open IndexReader, get OutOFMemory Exception

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Invalid
    • Affects Version/s: Lucene.Net 2.9.4g
    • Fix Version/s: None
    • Component/s: Lucene.Net Core
    • Labels:
      None
    • Environment:

      Windows server 2008R2

      Description

      Have build a large database with ~1Bn records (2 items per document) it has size 200GB on disk. I managed to write the indexe by chunking into 100,000 blocks as I ended up with some threading issues (another bug submission). Anyway the index is built but I can't open it and get a memory exception (process explorer gets to 1.5GB allocated before it dies but not sure how reliable that is, but do know there is plenty more RAM left on the box).
      Stack trace below:

      System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was
      thrown.
      at Lucene.Net.Index.TermInfosReader..ctor(Directory dir, String seg, FieldInf
      os fis, Int32 readBufferSize, Int32 indexDivisor)
      at Lucene.Net.Index.SegmentReader.CoreReaders..ctor(SegmentReader origInstanc
      e, Directory dir, SegmentInfo si, Int32 readBufferSize, Int32 termsIndexDivisor)

      at Lucene.Net.Index.SegmentReader.Get(Boolean readOnly, Directory dir, Segmen
      tInfo si, Int32 readBufferSize, Boolean doOpenStores, Int32 termInfosIndexDiviso
      r)
      at Lucene.Net.Index.SegmentReader.Get(Boolean readOnly, SegmentInfo si, Int32
      termInfosIndexDivisor)
      at Lucene.Net.Index.DirectoryReader..ctor(Directory directory, SegmentInfos s
      is, IndexDeletionPolicy deletionPolicy, Boolean readOnly, Int32 termInfosIndexDi
      visor)
      at Lucene.Net.Index.DirectoryReader.<>c_DisplayClass1.<Open>b_0(String segm
      entFileName)
      at Lucene.Net.Index.SegmentInfos.FindSegmentsFile.Run(IndexCommit commit)
      at Lucene.Net.Index.DirectoryReader.Open(Directory directory, IndexDeletionPo
      licy deletionPolicy, IndexCommit commit, Boolean readOnly, Int32 termInfosIndexD
      ivisor)
      at Lucene.Net.Index.IndexReader.Open(String path, Boolean readOnly)
      at Lucene.Net.Demo.SearchFiles.Main(String[] args)

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Steven
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development