Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5929

Standard highlighting doesn't work for ToParentBlockJoinQuery

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: modules/highlighter
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      Because WeightedSpanTermExtractor#extract doesn't check for ToParentBlockJoinQuery, the Highlighter class fails to produce highlights for this type of query.

      At first it may seem like there's no issue, because ToParentBlockJoinQuery only returns parent documents, while the highlighting applies to children. But if a client can directly supply the text from child documents (as elasticsearch does if _source is enabled), then highlighting will unexpectedly fail.

      A test case that triggers the bug is attached. The same issue exists for ToChildBlockJoinQuery.

      1. HighligherTest.patch
        6 kB
        Julie Tibshirani
      2. LUCENE-5929.patch
        11 kB
        Michael McCandless
      3. LUCENE-5929.patch
        9 kB
        Julie Tibshirani

        Activity

        Hide
        jtibs Julie Tibshirani added a comment - - edited

        I uploaded a patch that simply adds a check in WeightedSpanTermExtractor for ToParentBlockJoinQuery and ToChildBlockJoinQuery. I'm new to committing to Lucene – any suggestions would be much appreciated!

        Show
        jtibs Julie Tibshirani added a comment - - edited I uploaded a patch that simply adds a check in WeightedSpanTermExtractor for ToParentBlockJoinQuery and ToChildBlockJoinQuery. I'm new to committing to Lucene – any suggestions would be much appreciated!
        Hide
        jtibs Julie Tibshirani added a comment -

        Pinging this ticket since I haven't heard back in a while.

        Show
        jtibs Julie Tibshirani added a comment - Pinging this ticket since I haven't heard back in a while.
        Hide
        mikemccand Michael McCandless added a comment -

        Thanks Julie, the patch looks good! Do we really need the added join dep in benchmark?

        I updated the patch based on recent trunk changes, attached.

        However, "ant clean test" from lucene subdir fails, because ant doesn't build join before building highlight ... not sure yet where to fix this. If you manually "ant jar" from inside "lucene/join" first, it works.

        Show
        mikemccand Michael McCandless added a comment - Thanks Julie, the patch looks good! Do we really need the added join dep in benchmark? I updated the patch based on recent trunk changes, attached. However, "ant clean test" from lucene subdir fails, because ant doesn't build join before building highlight ... not sure yet where to fix this. If you manually "ant jar" from inside "lucene/join" first, it works.
        Hide
        mikemccand Michael McCandless added a comment -

        OK the added dep in benchmark is needed, because it depends on highlighter.

        And I made a local fix for "ant clean test" to work.

        Would be nice to also fix postings highlighter to handle the block join queries, but we can take that up separately... I'll commit shortly.

        Show
        mikemccand Michael McCandless added a comment - OK the added dep in benchmark is needed, because it depends on highlighter. And I made a local fix for "ant clean test" to work. Would be nice to also fix postings highlighter to handle the block join queries, but we can take that up separately... I'll commit shortly.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1639371 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1639371 ]

        LUCENE-5929: highlight terms from block join queries too

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1639371 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1639371 ] LUCENE-5929 : highlight terms from block join queries too
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1639372 from Michael McCandless in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1639372 ]

        LUCENE-5929: highlight terms from block join queries too

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1639372 from Michael McCandless in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1639372 ] LUCENE-5929 : highlight terms from block join queries too
        Hide
        mikemccand Michael McCandless added a comment -

        Thanks Julie!

        Show
        mikemccand Michael McCandless added a comment - Thanks Julie!
        Hide
        jtibs Julie Tibshirani added a comment -

        No problem, thank you Michael!

        Show
        jtibs Julie Tibshirani added a comment - No problem, thank you Michael!
        Hide
        anshumg Anshum Gupta added a comment -

        Bulk close after 5.0 release.

        Show
        anshumg Anshum Gupta added a comment - Bulk close after 5.0 release.
        Hide
        mikichi Michael Solomon added a comment -

        I have solr 5.4.1 but I have this problem too... what can I do about that?
        Thanks you,
        Michael
        P.s: I tried in the mailing list first: http://lucene.472066.n3.nabble.com/Fwd-Standard-highlighting-doesn-t-work-for-Block-Join-td4260784.html

        Show
        mikichi Michael Solomon added a comment - I have solr 5.4.1 but I have this problem too... what can I do about that? Thanks you, Michael P.s: I tried in the mailing list first: http://lucene.472066.n3.nabble.com/Fwd-Standard-highlighting-doesn-t-work-for-Block-Join-td4260784.html
        Hide
        vstrugatsky Vladimir Strugatsky added a comment -

        Michael Solomon - My understanding is that the fields you are highlighting need to be in a parent document, and that there is no support for highlighting fields in a child document. This JIRA fixed a bug with Highlighting of parent fields when using a Block Join Parser.

        Show
        vstrugatsky Vladimir Strugatsky added a comment - Michael Solomon - My understanding is that the fields you are highlighting need to be in a parent document, and that there is no support for highlighting fields in a child document. This JIRA fixed a bug with Highlighting of parent fields when using a Block Join Parser.

          People

          • Assignee:
            mikemccand Michael McCandless
            Reporter:
            jtibs Julie Tibshirani
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development