Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-4738

Killed JVM when first commit was running will generate a corrupted index

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.0
    • Fix Version/s: 4.3, 6.0
    • Component/s: core/index
    • Labels:
      None
    • Environment:

      OS: Linux 2.6.32-220.23.1.el6.x86_64
      Java: java version "1.7.0_05"
      Lucene: lucene-core-4.0.0

    • Lucene Fields:
      New

      Description

      1. Start a NEW IndexWriterBuilder on an empty folder,
      add some documents to the index
      2. Call commit
      3. When the segments_1 file with 0 byte was created, kill the JVM

      We will end with a corrupted index with an empty segments_1.

      We only have issue with the first commit crash.

      Also, if you tried to open an IndexSearcher on a new index. And the first commit on the index was not finished yet. Then you will see exception like:
      ===========================================================================
      org.apache.lucene.index.IndexNotFoundException: no segments* file found in org.apache.lucene.store.MMapDirectory@C:\tmp\testdir lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ee00df: files: [write.lock, _0.fdt, _0.fdx]
      at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:741)
      at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
      at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:65)
      ===========================================================================

      So when a new index was created, we should first create an empty index. We should not wait for the commit/close call to create the segment file.
      If we had an empty index there. It won't leave a corrupted index when there were a power issue on the first commit.
      And a concurrent IndexSearcher can access to the index(No match is better than exception).

        Attachments

        1. LUCENE-4738_test.patch
          1 kB
          Robert Muir
        2. LUCENE-4738.patch
          5 kB
          Michael McCandless
        3. LUCENE-4738.patch
          26 kB
          Michael McCandless
        4. LUCENE-4738.patch
          25 kB
          Michael McCandless

          Activity

            People

            • Assignee:
              mikemccand Michael McCandless
              Reporter:
              billowgao Billow Gao
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: