Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-485

IndexWriter.mergeSegments should not hold the commit lock while cleaning up.

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: core/index
    • Labels:
      None
    • Environment:

      checked in revision 367361

      Description

      Same happens in IndexWriter.addIndexes(IndexReader[] readers).

      The commit lock should be obtained whenever the Index structure/version is read or written. It should be kept for as short a period as possible.

      The write lock is needed to make sure only one IndexWriter or IndexReader instance can update the index (multiple IndexReaders can of course use the index for searching).

      The list of files that can be deleted is stored in the file "deletable". It is only read or written by the IndexWriter instance that holds the write lock, so there's no need to have the commit lock to to update it.

      On my production system deleting the obsolete segment files after a mergeSegments() happens can occasionally take several seconds and the commit lock blocks the searcher machines from updating their IndexReader instance.
      Even on a standalone machine, the time to update the segments file is about 3ms, the time to delete the obsolete segments about 30ms.

        Attachments

        1. LUCENE-485.patch
          2 kB
          Luc Vanlerberghe

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              lvl Luc Vanlerberghe
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: