Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7562

CompletionFieldsConsumer throws NPE on ghost fields

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.0, 5.5.4, 6.4
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      If you index SuggestField for some field X, but later delete all documents with that field, it can cause a ghost situation where the field infos believes field X exists yet the postings do not.

      I believe this bug is the root cause of this ES issue: https://github.com/elastic/elasticsearch/issues/21500

      1. LUCENE-7562.patch
        6 kB
        Michael McCandless

        Activity

        Hide
        mikemccand Michael McCandless added a comment -

        Patch against 6.2.x; I think it's ready.

        The fix itself is trivial (missing null check); the test was tricky!

        Show
        mikemccand Michael McCandless added a comment - Patch against 6.2.x; I think it's ready. The fix itself is trivial (missing null check); the test was tricky!
        Hide
        jpountz Adrien Grand added a comment -

        I see BasePostingsFormatTestCase has a test case for ghost fields, maybe we should have a test that extends BasePostingsFormatTestCase with CompletionPostingsFormat?

        Show
        jpountz Adrien Grand added a comment - I see BasePostingsFormatTestCase has a test case for ghost fields, maybe we should have a test that extends BasePostingsFormatTestCase with CompletionPostingsFormat?
        Hide
        mikemccand Michael McCandless added a comment -

        Thanks Adrien Grand I'll do that.

        Show
        mikemccand Michael McCandless added a comment - Thanks Adrien Grand I'll do that.
        Hide
        mikemccand Michael McCandless added a comment -

        Alas, that won't work (well) because `CompletionPostingsFormat` is not general purpose, e.g. it requires fields are indexed with positions/payloads, as SuggestField does ...

        Show
        mikemccand Michael McCandless added a comment - Alas, that won't work (well) because `CompletionPostingsFormat` is not general purpose, e.g. it requires fields are indexed with positions/payloads, as SuggestField does ...
        Hide
        jpountz Adrien Grand added a comment -

        Nevermind then, +1 to the current patch!

        Show
        jpountz Adrien Grand added a comment - Nevermind then, +1 to the current patch!
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 6ec68ddbe07804663fef3a4b8a849e8bacf20d64 in lucene-solr's branch refs/heads/master from Mike McCandless
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6ec68dd ]

        LUCENE-7562: don't throw NPE when encountering a level 2 ghost field

        Show
        jira-bot ASF subversion and git services added a comment - Commit 6ec68ddbe07804663fef3a4b8a849e8bacf20d64 in lucene-solr's branch refs/heads/master from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6ec68dd ] LUCENE-7562 : don't throw NPE when encountering a level 2 ghost field
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 42459a036d4fb5077b069797570f06318ce47929 in lucene-solr's branch refs/heads/branch_6x from Mike McCandless
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=42459a0 ]

        LUCENE-7562: don't throw NPE when encountering a level 2 ghost field

        Show
        jira-bot ASF subversion and git services added a comment - Commit 42459a036d4fb5077b069797570f06318ce47929 in lucene-solr's branch refs/heads/branch_6x from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=42459a0 ] LUCENE-7562 : don't throw NPE when encountering a level 2 ghost field
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit da641ba2da1675538176767d2d84bcf3cb5c4f21 in lucene-solr's branch refs/heads/branch_5_5 from Mike McCandless
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=da641ba ]

        LUCENE-7562: don't throw NPE when encountering a level 2 ghost field

        Show
        jira-bot ASF subversion and git services added a comment - Commit da641ba2da1675538176767d2d84bcf3cb5c4f21 in lucene-solr's branch refs/heads/branch_5_5 from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=da641ba ] LUCENE-7562 : don't throw NPE when encountering a level 2 ghost field

          People

          • Assignee:
            mikemccand Michael McCandless
            Reporter:
            mikemccand Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development