Lucene - Core
  1. Lucene - Core
  2. LUCENE-6505

NRT readers don't always reflect last commit

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.2, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Two cases here:

      • When I pull an NRT reader from IW,
        IR.getIndexCommit().getSegmentsFileName() should reflect what was
        last committed, but doesn't now
      • If I call IW.commit(), or IW.setCommitData(), but make no other
        changes, and then open a new NRT reader, I think it should reflect
        the new commit, but doesn't now
      1. LUCENE-6505.patch
        7 kB
        Michael McCandless

        Activity

        Hide
        Michael McCandless added a comment -

        Patch, with tests and fixes for these cases.

        I explored removing SegmentInfos.lastGeneration (failing to update
        that was the cause of the first issue) but it makes me nervous because
        on a non-tragic exception during commit it could mean that IW's
        segmentInfos, and any subsequently opened NRT reader, points to a
        non-existent (future) segments file, and could maybe also mean that
        IFD deletes the still-live one.

        I also tweaked infoStream logging a bit so IW says when it wrote the
        pending_segments_N, and is more timely saying when it wrote the final
        segments_N (so it no longer looks like it removed the old one before
        writing the new one!).

        Show
        Michael McCandless added a comment - Patch, with tests and fixes for these cases. I explored removing SegmentInfos.lastGeneration (failing to update that was the cause of the first issue) but it makes me nervous because on a non-tragic exception during commit it could mean that IW's segmentInfos, and any subsequently opened NRT reader, points to a non-existent (future) segments file, and could maybe also mean that IFD deletes the still-live one. I also tweaked infoStream logging a bit so IW says when it wrote the pending_segments_N, and is more timely saying when it wrote the final segments_N (so it no longer looks like it removed the old one before writing the new one!).
        Hide
        Robert Muir added a comment -

        +1

        I explored removing SegmentInfos.lastGeneration

        Its too bad that didnt work out... for now it lives...

        Show
        Robert Muir added a comment - +1 I explored removing SegmentInfos.lastGeneration Its too bad that didnt work out... for now it lives...
        Hide
        Michael McCandless added a comment -

        I'd like to backport for 5.2.0 RC2...

        Show
        Michael McCandless added a comment - I'd like to backport for 5.2.0 RC2...
        Hide
        ASF subversion and git services added a comment -

        Commit 1682296 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1682296 ]

        LUCENE-6505: NRT readers now reflect prior commit metadata

        Show
        ASF subversion and git services added a comment - Commit 1682296 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1682296 ] LUCENE-6505 : NRT readers now reflect prior commit metadata
        Hide
        ASF subversion and git services added a comment -

        Commit 1682299 from Michael McCandless in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1682299 ]

        LUCENE-6505: NRT readers now reflect prior commit metadata

        Show
        ASF subversion and git services added a comment - Commit 1682299 from Michael McCandless in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1682299 ] LUCENE-6505 : NRT readers now reflect prior commit metadata
        Hide
        ASF subversion and git services added a comment -

        Commit 1682301 from Michael McCandless in branch 'dev/branches/lucene_solr_5_2'
        [ https://svn.apache.org/r1682301 ]

        LUCENE-6505: NRT readers now reflect prior commit metadata

        Show
        ASF subversion and git services added a comment - Commit 1682301 from Michael McCandless in branch 'dev/branches/lucene_solr_5_2' [ https://svn.apache.org/r1682301 ] LUCENE-6505 : NRT readers now reflect prior commit metadata
        Hide
        Anshum Gupta added a comment -

        Bulk close for 5.2.0.

        Show
        Anshum Gupta added a comment - Bulk close for 5.2.0.

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development