Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8754

SegmentInfo#toString can cause ConcurrentModificationException

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 8.1, 9.0
    • None
    • None
    • New

    Description

      A recent change increased the likelihood for this issue to show up but it can already happen before since we are using the attributes map in the StoredFieldsFormat for quite some time. I found this issue due to a test failure on our CI:

      13:11:56    [junit4] Suite: org.apache.lucene.index.TestIndexSorting
      13:11:56    [junit4]   2> apr 05, 2019 8:11:53 AM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
      13:11:56    [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-507,5,TGRP-TestIndexSorting]
      13:11:56    [junit4]   2> java.util.ConcurrentModificationException
      13:11:56    [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([7C25B308F180203B]:0)
      13:11:56    [junit4]   2> 	at java.util.HashMap$HashIterator.nextNode(HashMap.java:1442)
      13:11:56    [junit4]   2> 	at java.util.HashMap$EntryIterator.next(HashMap.java:1476)
      13:11:56    [junit4]   2> 	at java.util.HashMap$EntryIterator.next(HashMap.java:1474)
      13:11:56    [junit4]   2> 	at java.util.AbstractMap.toString(AbstractMap.java:554)
      13:11:56    [junit4]   2> 	at org.apache.lucene.index.SegmentInfo.toString(SegmentInfo.java:222)
      13:11:56    [junit4]   2> 	at org.apache.lucene.index.SegmentCommitInfo.toString(SegmentCommitInfo.java:345)
      13:11:56    [junit4]   2> 	at org.apache.lucene.index.SegmentCommitInfo.toString(SegmentCommitInfo.java:364)
      13:11:56    [junit4]   2> 	at java.lang.String.valueOf(String.java:2994)
      13:11:56    [junit4]   2> 	at java.lang.StringBuilder.append(StringBuilder.java:131)
      13:11:56    [junit4]   2> 	at java.util.AbstractMap.toString(AbstractMap.java:557)
      13:11:56    [junit4]   2> 	at java.util.Collections$UnmodifiableMap.toString(Collections.java:1493)
      13:11:56    [junit4]   2> 	at java.lang.String.valueOf(String.java:2994)
      13:11:56    [junit4]   2> 	at java.lang.StringBuilder.append(StringBuilder.java:131)
      13:11:56    [junit4]   2> 	at org.apache.lucene.index.TieredMergePolicy.findForcedMerges(TieredMergePolicy.java:628)
      13:11:56    [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updatePendingMerges(IndexWriter.java:2181)
      13:11:56    [junit4]   2> 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2154)
      13:11:56    [junit4]   2> 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1988)
      13:11:56    [junit4]   2> 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1939)
      13:11:56    [junit4]   2> 	at org.apache.lucene.index.TestIndexSorting$UpdateRunnable.run(TestIndexSorting.java:1851)
      13:11:56    [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
      13:11:56    [junit4]   2> 
      13:11:56    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexSorting -Dtests.method=testConcurrentUpdates -Dtests.seed=7C25B308F180203B -Dtests.slow=true -Dtest
      

      The issue is that we update the attributes map (also we similarly do the same for diagnostics but it's not necessarily causing the issue since the diagnostics map is never modified) during the merge process but access it in the merge policy when looking at running merges and there we call toString on SegmentCommitInfo which happens without any synchronization. This is technically unsafe publication but IW is a mess along those lines and real fixes would require significant changes.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            simonw Simon Willnauer
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3h
                3h

                Slack

                  Issue deployment