Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8045

ParallelReader does not propagate doc values generation numbers

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 7.2
    • None
    • None
    • New

    Description

      Exposed by this test failure: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/777/testReport/junit/org.apache.lucene.search/TestLRUQueryCache/testDocValuesUpdatesDontBreakCache/

      A reader is randomly wrapped with a ParallelLeafReader, which does not then correctly propagate the dvGen into its own FieldInfo.

      Attachments

        1. LUCENE-8045.patch
          5 kB
          Alan Woodward

        Activity

          romseygeek Alan Woodward added a comment -

          Looking further, I think the FieldInfos.Builder behaviour is correct, as it's used by SegmentMerger and ReadersAndUpdates, where deletes and DV updates are flattened out. So this is really a bug in ParallelReader.

          romseygeek Alan Woodward added a comment - Looking further, I think the FieldInfos.Builder behaviour is correct, as it's used by SegmentMerger and ReadersAndUpdates, where deletes and DV updates are flattened out. So this is really a bug in ParallelReader.
          romseygeek Alan Woodward added a comment -

          Thinking about this more, ParallelReader is really set up as an alternative to updatedDocValues, so maybe the simplest thing here is to disallow them completely? Check in the constructor if the incoming FieldInfo has a dvGen > -1, and throw an exception if so.

          romseygeek Alan Woodward added a comment - Thinking about this more, ParallelReader is really set up as an alternative to updatedDocValues, so maybe the simplest thing here is to disallow them completely? Check in the constructor if the incoming FieldInfo has a dvGen > -1, and throw an exception if so.
          romseygeek Alan Woodward added a comment -

          Turns out that it wasn't difficult to make ParallelLeafReader.getFieldInfos() return the right thing, and dv updates do seem to work correctly otherwise. Here's a patch fixing things.

          romseygeek Alan Woodward added a comment - Turns out that it wasn't difficult to make ParallelLeafReader.getFieldInfos() return the right thing, and dv updates do seem to work correctly otherwise. Here's a patch fixing things.

          Commit c0ed9199fae03142b4fe6adebdccec728158eb5b in lucene-solr's branch refs/heads/branch_7x from romseygeek
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c0ed919 ]

          LUCENE-8045: ParallelLeafReader should correctly report FieldInfo.dvGen

          jira-bot ASF subversion and git services added a comment - Commit c0ed9199fae03142b4fe6adebdccec728158eb5b in lucene-solr's branch refs/heads/branch_7x from romseygeek [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c0ed919 ] LUCENE-8045 : ParallelLeafReader should correctly report FieldInfo.dvGen

          Commit f0fec1fc5f037ed18c901e43f1d17c4e6594f152 in lucene-solr's branch refs/heads/branch_7x from romseygeek
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f0fec1f ]

          Revert "LUCENE-8017: Don't use ParallelReader in tests"

          This reverts commit ff4874f3d3ff6c307121a6a1f6d87a33d45a48a4.

          LUCENE-8045 makes this unnecessary

          jira-bot ASF subversion and git services added a comment - Commit f0fec1fc5f037ed18c901e43f1d17c4e6594f152 in lucene-solr's branch refs/heads/branch_7x from romseygeek [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f0fec1f ] Revert " LUCENE-8017 : Don't use ParallelReader in tests" This reverts commit ff4874f3d3ff6c307121a6a1f6d87a33d45a48a4. LUCENE-8045 makes this unnecessary

          Commit 77ea2110965a05e91cbd04257ccd394e1fd3fad3 in lucene-solr's branch refs/heads/master from romseygeek
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=77ea211 ]

          LUCENE-8045: ParallelLeafReader should correctly report FieldInfo.dvGen

          jira-bot ASF subversion and git services added a comment - Commit 77ea2110965a05e91cbd04257ccd394e1fd3fad3 in lucene-solr's branch refs/heads/master from romseygeek [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=77ea211 ] LUCENE-8045 : ParallelLeafReader should correctly report FieldInfo.dvGen

          Commit e827f17be59d6f505cd920756e3ce780d30e2eb2 in lucene-solr's branch refs/heads/master from romseygeek
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e827f17 ]

          Revert "LUCENE-8017: Don't use ParallelReader in tests"

          This reverts commit ff4874f3d3ff6c307121a6a1f6d87a33d45a48a4.

          LUCENE-8045 makes this unnecessary

          jira-bot ASF subversion and git services added a comment - Commit e827f17be59d6f505cd920756e3ce780d30e2eb2 in lucene-solr's branch refs/heads/master from romseygeek [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e827f17 ] Revert " LUCENE-8017 : Don't use ParallelReader in tests" This reverts commit ff4874f3d3ff6c307121a6a1f6d87a33d45a48a4. LUCENE-8045 makes this unnecessary
          tomoko Tomoko Uchida added a comment -

          This issue was moved to GitHub issue: #9093.

          tomoko Tomoko Uchida added a comment - This issue was moved to GitHub issue: #9093 .

          People

            romseygeek Alan Woodward
            romseygeek Alan Woodward
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: