Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8048

Filesystems do not guarantee order of directories updates

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 7.2
    • None
    • None
    • New

    Description

      Currently when index is written to disk the following sequence of events is taking place:

      • write segment file
      • sync segment file
      • write segment file
      • sync segment file
        ...
      • write list of segments
      • sync list of segments
      • rename list of segments
      • sync index directory

      This sequence leads to potential window of opportunity for system to crash after 'rename list of segments' but before 'sync index directory' and depending on exact filesystem implementation this may potentially lead to 'list of segments' being visible in directory while some of the segments are not.

      Solution to this is to sync index directory after all segments have been written. This commit shows idea implemented. I'm fairly certain that I didn't find all the places this may be potentially happening.

      Attachments

        1. LUCENE-8048.patch
          6 kB
          Erick Erickson
        2. LUCENE-8048.patch
          5 kB
          Simon Willnauer
        3. LUCENE-8048.patch
          0.6 kB
          Erick Erickson
        4. Screen Shot 2017-11-22 at 12.34.51 PM.png
          86 kB
          Erick Erickson

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            erickerickson Erick Erickson
            mar-kolya Nikolay Martynov
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment