Solr
  1. Solr
  2. SOLR-2390

Performance of usePhraseHighlighter is terrible on very large Documents, regardless of hl.maxDocCharsToAnalyze

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2, 4.0-ALPHA
    • Component/s: highlighter
    • Labels:
      None

      Description

      There is a large performance bug here.

        Issue Links

          Activity

          Hide
          Mark Miller added a comment -

          This fix makes highlighting a results page drop from 20 some seconds for me to like 300ms with the default hl.maxDocCharsToAnalyze setting and very large documents.

          Show
          Mark Miller added a comment - This fix makes highlighting a results page drop from 20 some seconds for me to like 300ms with the default hl.maxDocCharsToAnalyze setting and very large documents.
          Hide
          Robert Muir added a comment -

          not sure we should set this to 3.1, i don't even see a patch.

          Show
          Robert Muir added a comment - not sure we should set this to 3.1, i don't even see a patch.
          Hide
          Mark Miller added a comment -

          i don't even see a patch

          So what? Take a look at LUCENE-2939 again - that patch spans Lucene and Solr.

          Show
          Mark Miller added a comment - i don't even see a patch So what? Take a look at LUCENE-2939 again - that patch spans Lucene and Solr.
          Hide
          Mark Miller added a comment -

          Did you really kick it out of 3.1 on Grant without even trying to pay attention to the issue?

          This is obviously the same issue as LUCENE-2939, but tracking the Solr part of the bug.

          Show
          Mark Miller added a comment - Did you really kick it out of 3.1 on Grant without even trying to pay attention to the issue? This is obviously the same issue as LUCENE-2939 , but tracking the Solr part of the bug.
          Hide
          Robert Muir added a comment -

          I didn't kick it out, its an issue with no patch that just this morning
          at the last minute got shoved into 3.1.. and its not marked as blocker.

          Show
          Robert Muir added a comment - I didn't kick it out, its an issue with no patch that just this morning at the last minute got shoved into 3.1.. and its not marked as blocker.
          Hide
          Robert Muir added a comment -

          You can go back and forth on this if you want.

          Its too late, its not going to make it.

          Show
          Robert Muir added a comment - You can go back and forth on this if you want. Its too late, its not going to make it.
          Hide
          Mark Miller added a comment -

          That's up for discussion King robert.

          Show
          Mark Miller added a comment - That's up for discussion King robert.
          Hide
          Robert Muir added a comment -

          Not really: its clearly documented: only an issue marked blocker can delay a release candidate build: http://wiki.apache.org/lucene-java/ReleaseTodo

          I think its totally natural to expect me to try to push back on last minute stuff getting shoved into the release, especially if these were not INTRODUCED in 3.1, don't have any patches at all, only refer to a misconfigured case where someone didn't index their content correctly, and could be fixed in 3.2 instead.

          Show
          Robert Muir added a comment - Not really: its clearly documented: only an issue marked blocker can delay a release candidate build: http://wiki.apache.org/lucene-java/ReleaseTodo I think its totally natural to expect me to try to push back on last minute stuff getting shoved into the release, especially if these were not INTRODUCED in 3.1, don't have any patches at all, only refer to a misconfigured case where someone didn't index their content correctly, and could be fixed in 3.2 instead.
          Hide
          Mark Miller added a comment -

          Not really: its clearly documented: only an issue marked blocker can delay a release candidate build: http://wiki.apache.org/lucene-java/ReleaseTodo

          This is not a rule. Nor an excuse to just flip back something someone changed with 0 discussion.

          I think its totally natural to expect me to try to push back on last minute stuff getting shoved into the release

          So push back - don't revert someones change 2 seconds after they made it.

          especially if these were not INTRODUCED in 3.1, don't have any patches at all, only refer to a misconfigured case where someone didn't index their content correctly, and could be fixed in 3.2 instead.

          Huh!? LUCENE-2939 was marked 3.1 last night. This is that issue if you cannot tell. They should both be marked the same in either case.

          Show
          Mark Miller added a comment - Not really: its clearly documented: only an issue marked blocker can delay a release candidate build: http://wiki.apache.org/lucene-java/ReleaseTodo This is not a rule. Nor an excuse to just flip back something someone changed with 0 discussion. I think its totally natural to expect me to try to push back on last minute stuff getting shoved into the release So push back - don't revert someones change 2 seconds after they made it. especially if these were not INTRODUCED in 3.1, don't have any patches at all, only refer to a misconfigured case where someone didn't index their content correctly, and could be fixed in 3.2 instead. Huh!? LUCENE-2939 was marked 3.1 last night. This is that issue if you cannot tell. They should both be marked the same in either case.
          Hide
          Mark Miller added a comment -

          only refer to a misconfigured case where someone didn't index their content correctly

          This shows you don't understand the issue.

          Show
          Mark Miller added a comment - only refer to a misconfigured case where someone didn't index their content correctly This shows you don't understand the issue.
          Hide
          Robert Muir added a comment -

          Huh!? LUCENE-2939 was marked 3.1 last night. This is that issue if you cannot tell. They should both be marked the same in either case.

          What version of Lucene was the performance bug introduced in?

          Show
          Robert Muir added a comment - Huh!? LUCENE-2939 was marked 3.1 last night. This is that issue if you cannot tell. They should both be marked the same in either case. What version of Lucene was the performance bug introduced in?
          Hide
          Robert Muir added a comment -

          This shows you don't understand the issue.

          Maybe thats true, even more the reason to push back. As I see it we provide the fast-vector-highlighter option that is a viable user workaround for the issue.

          Show
          Robert Muir added a comment - This shows you don't understand the issue. Maybe thats true, even more the reason to push back. As I see it we provide the fast-vector-highlighter option that is a viable user workaround for the issue.
          Hide
          Robert Muir added a comment -

          Bulk close for 3.2

          Show
          Robert Muir added a comment - Bulk close for 3.2

            People

            • Assignee:
              Mark Miller
              Reporter:
              Mark Miller
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development