Lucene - Core
  1. Lucene - Core
  2. LUCENE-5615

validate per-segment delete counts on write

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.8, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      When reading the segment infos, we already check if the delCount is out of bounds for a segment (< 0 or > docCount) and throw CorruptIndexException if so.

      I think we should catch this when writing as well, in case there are Lucene bugs that mess up the delete counts. We have just an assert today...

      1. LUCENE-5615.4x.patch
        4 kB
        Michael McCandless
      2. LUCENE-5615.patch
        3 kB
        Michael McCandless

        Activity

        Hide
        Michael McCandless added a comment -

        Simple patch, tests pass. I also upgraded SegmentCommitInfo.setDelCount's assert to a real check.

        Show
        Michael McCandless added a comment - Simple patch, tests pass. I also upgraded SegmentCommitInfo.setDelCount's assert to a real check.
        Hide
        Robert Muir added a comment -

        +1

        Is there similar safety we can add to the scary 3.x upgraded si in 4.x? Just an idea...

        Show
        Robert Muir added a comment - +1 Is there similar safety we can add to the scary 3.x upgraded si in 4.x? Just an idea...
        Hide
        Michael McCandless added a comment -

        Patch for 4.x.

        Good idea Rob: it'd be good to do some validation when we see a 3.x segment that's being upgraded ... those files are clearly "oldish" and have more chance of corruption.

        I upgraded one assert in write3xInfo (verifying codec is correct) to a real check. The only other thing I thought of is we could assert the files referenced by the SI in fact exist (I haven't done that yet)? Anything else?

        Show
        Michael McCandless added a comment - Patch for 4.x. Good idea Rob: it'd be good to do some validation when we see a 3.x segment that's being upgraded ... those files are clearly "oldish" and have more chance of corruption. I upgraded one assert in write3xInfo (verifying codec is correct) to a real check. The only other thing I thought of is we could assert the files referenced by the SI in fact exist (I haven't done that yet)? Anything else?
        Hide
        Uwe Schindler added a comment -

        +1 - Nuke the asserts and make real checks out of them! Those simple ones cost nothing!

        Show
        Uwe Schindler added a comment - +1 - Nuke the asserts and make real checks out of them! Those simple ones cost nothing!
        Hide
        Robert Muir added a comment -

        +1

        Show
        Robert Muir added a comment - +1
        Hide
        ASF subversion and git services added a comment -

        Commit 1588459 from mikemccand@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1588459 ]

        LUCENE-5615: catch invalid per-segment delete counts when writing the segment

        Show
        ASF subversion and git services added a comment - Commit 1588459 from mikemccand@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1588459 ] LUCENE-5615 : catch invalid per-segment delete counts when writing the segment
        Hide
        ASF subversion and git services added a comment -

        Commit 1588464 from mikemccand@apache.org in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1588464 ]

        LUCENE-5615: catch invalid per-segment delete counts when writing the segment

        Show
        ASF subversion and git services added a comment - Commit 1588464 from mikemccand@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1588464 ] LUCENE-5615 : catch invalid per-segment delete counts when writing the segment
        Hide
        ASF subversion and git services added a comment -

        Commit 1588466 from mikemccand@apache.org in branch 'dev/branches/lucene_solr_4_8'
        [ https://svn.apache.org/r1588466 ]

        LUCENE-5615: catch invalid per-segment delete counts when writing the segment

        Show
        ASF subversion and git services added a comment - Commit 1588466 from mikemccand@apache.org in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1588466 ] LUCENE-5615 : catch invalid per-segment delete counts when writing the segment
        Hide
        Uwe Schindler added a comment -

        Close issue after release of 4.8.0

        Show
        Uwe Schindler added a comment - Close issue after release of 4.8.0

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development