Lucene - Core
  1. Lucene - Core
  2. LUCENE-5929

Standard highlighting doesn't work for ToParentBlockJoinQuery

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: modules/highlighter
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      Because WeightedSpanTermExtractor#extract doesn't check for ToParentBlockJoinQuery, the Highlighter class fails to produce highlights for this type of query.

      At first it may seem like there's no issue, because ToParentBlockJoinQuery only returns parent documents, while the highlighting applies to children. But if a client can directly supply the text from child documents (as elasticsearch does if _source is enabled), then highlighting will unexpectedly fail.

      A test case that triggers the bug is attached. The same issue exists for ToChildBlockJoinQuery.

      1. HighligherTest.patch
        6 kB
        Julie Tibshirani
      2. LUCENE-5929.patch
        11 kB
        Michael McCandless
      3. LUCENE-5929.patch
        9 kB
        Julie Tibshirani

        Activity

        Hide
        Julie Tibshirani added a comment - - edited

        I uploaded a patch that simply adds a check in WeightedSpanTermExtractor for ToParentBlockJoinQuery and ToChildBlockJoinQuery. I'm new to committing to Lucene – any suggestions would be much appreciated!

        Show
        Julie Tibshirani added a comment - - edited I uploaded a patch that simply adds a check in WeightedSpanTermExtractor for ToParentBlockJoinQuery and ToChildBlockJoinQuery. I'm new to committing to Lucene – any suggestions would be much appreciated!
        Hide
        Julie Tibshirani added a comment -

        Pinging this ticket since I haven't heard back in a while.

        Show
        Julie Tibshirani added a comment - Pinging this ticket since I haven't heard back in a while.
        Hide
        Michael McCandless added a comment -

        Thanks Julie, the patch looks good! Do we really need the added join dep in benchmark?

        I updated the patch based on recent trunk changes, attached.

        However, "ant clean test" from lucene subdir fails, because ant doesn't build join before building highlight ... not sure yet where to fix this. If you manually "ant jar" from inside "lucene/join" first, it works.

        Show
        Michael McCandless added a comment - Thanks Julie, the patch looks good! Do we really need the added join dep in benchmark? I updated the patch based on recent trunk changes, attached. However, "ant clean test" from lucene subdir fails, because ant doesn't build join before building highlight ... not sure yet where to fix this. If you manually "ant jar" from inside "lucene/join" first, it works.
        Hide
        Michael McCandless added a comment -

        OK the added dep in benchmark is needed, because it depends on highlighter.

        And I made a local fix for "ant clean test" to work.

        Would be nice to also fix postings highlighter to handle the block join queries, but we can take that up separately... I'll commit shortly.

        Show
        Michael McCandless added a comment - OK the added dep in benchmark is needed, because it depends on highlighter. And I made a local fix for "ant clean test" to work. Would be nice to also fix postings highlighter to handle the block join queries, but we can take that up separately... I'll commit shortly.
        Hide
        ASF subversion and git services added a comment -

        Commit 1639371 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1639371 ]

        LUCENE-5929: highlight terms from block join queries too

        Show
        ASF subversion and git services added a comment - Commit 1639371 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1639371 ] LUCENE-5929 : highlight terms from block join queries too
        Hide
        ASF subversion and git services added a comment -

        Commit 1639372 from Michael McCandless in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1639372 ]

        LUCENE-5929: highlight terms from block join queries too

        Show
        ASF subversion and git services added a comment - Commit 1639372 from Michael McCandless in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1639372 ] LUCENE-5929 : highlight terms from block join queries too
        Hide
        Michael McCandless added a comment -

        Thanks Julie!

        Show
        Michael McCandless added a comment - Thanks Julie!
        Hide
        Julie Tibshirani added a comment -

        No problem, thank you Michael!

        Show
        Julie Tibshirani added a comment - No problem, thank you Michael!
        Hide
        Anshum Gupta added a comment -

        Bulk close after 5.0 release.

        Show
        Anshum Gupta added a comment - Bulk close after 5.0 release.
        Hide
        Michael Solomon added a comment -

        I have solr 5.4.1 but I have this problem too... what can I do about that?
        Thanks you,
        Michael
        P.s: I tried in the mailing list first: http://lucene.472066.n3.nabble.com/Fwd-Standard-highlighting-doesn-t-work-for-Block-Join-td4260784.html

        Show
        Michael Solomon added a comment - I have solr 5.4.1 but I have this problem too... what can I do about that? Thanks you, Michael P.s: I tried in the mailing list first: http://lucene.472066.n3.nabble.com/Fwd-Standard-highlighting-doesn-t-work-for-Block-Join-td4260784.html

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Julie Tibshirani
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development