Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8328

ReadersAndUpdates#getLatestReader should execute under lock

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 7.4, 8.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      Patch Available

      Description

      It's possible for a merge thread to acquire an index reader which is closed before it can incRef.

      Merge stack trace:
      
      at __randomizedtesting.SeedInfo.seed([136983A068AA2F9D]:0)
      at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:257)
      at org.apache.lucene.index.IndexReader.incRef(IndexReader.java:184)
      at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:198)
      at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:728)
      at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4355)
      at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4043)
      at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
      at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2145)
      at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:542)
      at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288)
      at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263)
      at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253)
      at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
      
      Refresh stack trace:
      at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:238)
      at org.apache.lucene.index.ReadersAndUpdates.createNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:675)
      at org.apache.lucene.index.ReadersAndUpdates.swapNewReaderWithLatestLiveDocs(ReadersAndUpdates.java:686)
      at org.apache.lucene.index.ReadersAndUpdates.getLatestReader(ReadersAndUpdates.java:260)
      at org.elasticsearch.index.shard.ElasticsearchMergePolicy.keepFullyDeletedSegment(ElasticsearchMergePolicy.java:143)
      at org.apache.lucene.index.ReadersAndUpdates.keepFullyDeletedSegment(ReadersAndUpdates.java:769)
      at org.apache.lucene.index.IndexWriter.isFullyDeleted(IndexWriter.java:5124)
      at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3306)
      at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514)
      at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:288)
      at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:263)
      at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:253)
      at org.apache.lucene.index.FilterDirectoryReader.doOpenIfChanged(FilterDirectoryReader.java:104)
      at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)

      The problem is that `ReadersAndUpdates#getLatestReader` is executed concurrently without holding lock.

        Attachments

        1. LUCENE-8328.patch
          5 kB
          Nhat Nguyen
        2. LUCENE-8328.patch
          4 kB
          Nhat Nguyen

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dnhatn Nhat Nguyen
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: