Lucene - Core
  1. Lucene - Core
  2. LUCENE-485

IndexWriter.mergeSegments should not hold the commit lock while cleaning up.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: core/index
    • Labels:
      None
    • Environment:

      checked in revision 367361

      Description

      Same happens in IndexWriter.addIndexes(IndexReader[] readers).

      The commit lock should be obtained whenever the Index structure/version is read or written. It should be kept for as short a period as possible.

      The write lock is needed to make sure only one IndexWriter or IndexReader instance can update the index (multiple IndexReaders can of course use the index for searching).

      The list of files that can be deleted is stored in the file "deletable". It is only read or written by the IndexWriter instance that holds the write lock, so there's no need to have the commit lock to to update it.

      On my production system deleting the obsolete segment files after a mergeSegments() happens can occasionally take several seconds and the commit lock blocks the searcher machines from updating their IndexReader instance.
      Even on a standalone machine, the time to update the segments file is about 3ms, the time to delete the obsolete segments about 30ms.

      1. LUCENE-485.patch
        2 kB
        Luc Vanlerberghe

        Activity

          People

          • Assignee:
            Unassigned
            Reporter:
            Luc Vanlerberghe
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development