Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7581

IndexWriter#updateDocValues can break index sorting

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.0, 6.4
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      IndexWriter#updateDocValues can break index sorting if it is called on a field that is used in the index sorting specification.
      TestIndexSorting has a test for this case: #testConcurrentDVUpdates
      but only L1 merge are checked. Any LN merge would fail the test because the inner sort of the segment is not re-compute during/after DV updates.

      1. LUCENE-7581.patch
        7 kB
        Jim Ferenczi
      2. LUCENE-7581.patch
        3 kB
        Jim Ferenczi

        Activity

        Hide
        jim.ferenczi Jim Ferenczi added a comment -

        I attached a patch that fails the test if a second round of DV updates are run.

        Show
        jim.ferenczi Jim Ferenczi added a comment - I attached a patch that fails the test if a second round of DV updates are run.
        Hide
        mikemccand Michael McCandless added a comment -

        Thanks Jim Ferenczi; I think we need to fix the update DVs APIs to prevent changing any field involved in the index sort?

        Show
        mikemccand Michael McCandless added a comment - Thanks Jim Ferenczi ; I think we need to fix the update DVs APIs to prevent changing any field involved in the index sort?
        Hide
        jim.ferenczi Jim Ferenczi added a comment -

        Michael McCandless I think so too. I'll work on a patch.

        Show
        jim.ferenczi Jim Ferenczi added a comment - Michael McCandless I think so too. I'll work on a patch.
        Hide
        jim.ferenczi Jim Ferenczi added a comment -

        Here is a patch that fails DV updates on a field involved in the index sort.
        I also modified TestIndexSorting#testConcurrentDVUpdates which now test DV updates that are not involved in the index sort.

        Show
        jim.ferenczi Jim Ferenczi added a comment - Here is a patch that fails DV updates on a field involved in the index sort. I also modified TestIndexSorting#testConcurrentDVUpdates which now test DV updates that are not involved in the index sort.
        Hide
        mikemccand Michael McCandless added a comment -

        Thanks Jim Ferenczi, this looks great; I'll push shortly.

        Show
        mikemccand Michael McCandless added a comment - Thanks Jim Ferenczi , this looks great; I'll push shortly.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 4efbde4e76277f364952866c071bb953ca2be070 in lucene-solr's branch refs/heads/master from Mike McCandless
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4efbde4 ]

        LUCENE-7581: don't allow updating a doc values field if it's used in the index sort

        Show
        jira-bot ASF subversion and git services added a comment - Commit 4efbde4e76277f364952866c071bb953ca2be070 in lucene-solr's branch refs/heads/master from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4efbde4 ] LUCENE-7581 : don't allow updating a doc values field if it's used in the index sort
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 928fa91c894867f0543432e5036bb09615a6d7f1 in lucene-solr's branch refs/heads/branch_6x from Mike McCandless
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=928fa91 ]

        LUCENE-7581: don't allow updating a doc values field if it's used in the index sort

        Show
        jira-bot ASF subversion and git services added a comment - Commit 928fa91c894867f0543432e5036bb09615a6d7f1 in lucene-solr's branch refs/heads/branch_6x from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=928fa91 ] LUCENE-7581 : don't allow updating a doc values field if it's used in the index sort
        Hide
        mikemccand Michael McCandless added a comment -

        Thank you Jim Ferenczi!

        Show
        mikemccand Michael McCandless added a comment - Thank you Jim Ferenczi !
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 4efbde4e76277f364952866c071bb953ca2be070 in lucene-solr's branch refs/heads/feature/metrics from Mike McCandless
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4efbde4 ]

        LUCENE-7581: don't allow updating a doc values field if it's used in the index sort

        Show
        jira-bot ASF subversion and git services added a comment - Commit 4efbde4e76277f364952866c071bb953ca2be070 in lucene-solr's branch refs/heads/feature/metrics from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4efbde4 ] LUCENE-7581 : don't allow updating a doc values field if it's used in the index sort

          People

          • Assignee:
            Unassigned
            Reporter:
            jim.ferenczi Jim Ferenczi
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development