Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 5.2.1
    • Fix Version/s: 5.3, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      I expect the following test to pass, but it's failing in the latest Lucene 5.2.1:

      FailingTest.java
      public void testBlendedInfixSuggesterDedupsOnWeightTitleAndPayload() throws Exception {
      
      //Only the payload is different
          Input[] inputDocuments = new Input[]{
              new Input("lend me your ear", 7, new BytesRef("uid1")),
              new Input("lend me your ear", 7, new BytesRef("uid2")),
          };
      
          Analyzer a = new MockAnalyzer(random(), MockTokenizer.WHITESPACE, false);
          BlendedInfixSuggester suggester = new BlendedInfixSuggester(newDirectory(), a, a, AnalyzingInfixSuggester.DEFAULT_MIN_PREFIX_CHARS,
              BlendedInfixSuggester.BlenderType.POSITION_RECIPROCAL, 10, false);
      
          InputArrayIterator inputArrayIterator = new InputArrayIterator(inputDocuments);
          suggester.build(inputArrayIterator);
      
          List<Lookup.LookupResult> results = suggester.lookup(TestUtil.stringToCharSequence("ear", random()), 10, true, true);
      
          suggester.close();
          a.close();
      
          assertEquals(2, results.size());
      
        }
      
      

      This test is failing because the BlendedInfixSuggester internally uses a TreeSet for storing the results and the corresponding Comparator only uses text+weight meaning that results with different payloads are collapsed into one.

      Michael McCandless, The idea here is that if two ingested documents have the same title and weight, but different payloads, then they are two different things and folding them into a single document would mean loosing the payload information.

      1. LUCENE-6680.patch
        5 kB
        Arcadius Ahouansou

        Activity

        Hide
        Arcadius Ahouansou added a comment -

        Initial patch fixing the comparator to take into account the payload when everything else is same.

        Show
        Arcadius Ahouansou added a comment - Initial patch fixing the comparator to take into account the payload when everything else is same.
        Hide
        ASF subversion and git services added a comment -

        Commit 1691282 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1691282 ]

        LUCENE-6680: don't lose a suggestion that differs only in payload from another suggestion

        Show
        ASF subversion and git services added a comment - Commit 1691282 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1691282 ] LUCENE-6680 : don't lose a suggestion that differs only in payload from another suggestion
        Hide
        Michael McCandless added a comment -

        Thanks Arcadius Ahouansou, I just committed your patch with a small tweak to the if statement logic in the comparator...

        Show
        Michael McCandless added a comment - Thanks Arcadius Ahouansou , I just committed your patch with a small tweak to the if statement logic in the comparator...
        Hide
        ASF subversion and git services added a comment -

        Commit 1691283 from Michael McCandless in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1691283 ]

        LUCENE-6680: don't lose a suggestion that differs only in payload from another suggestion

        Show
        ASF subversion and git services added a comment - Commit 1691283 from Michael McCandless in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1691283 ] LUCENE-6680 : don't lose a suggestion that differs only in payload from another suggestion
        Hide
        Arcadius Ahouansou added a comment -

        Thank you very much Michael McCandless for helping get this issue fixed.

        Show
        Arcadius Ahouansou added a comment - Thank you very much Michael McCandless for helping get this issue fixed.
        Hide
        Shalin Shekhar Mangar added a comment -

        Bulk close for 5.3.0 release

        Show
        Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

          People

          • Assignee:
            Unassigned
            Reporter:
            Arcadius Ahouansou
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development