Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5660

AnalyzingSuggester needs reasonable limits on max suggestion length

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Issues like LUCENE-5628 LUCENE-5659 are hacks around the problem that the AnalyzingSuggester allows automata to "explode".

      I don't think we should try to hack up the automata lib to the point its un-understandable to support this, since its designed for smaller automata such as queries and is appropriate for that.

      We should instead just set appropriate limits out of box so AnalyzingSuggester doesnt blow up.

      Remember this is a suggester, it needs to be fast. The fact that the automata lib blows up on stupid unit tests or whatever are doing this, instead of being silently slow, is a good thing.

      1. LUCENE-5660.patch
        3 kB
        Michael McCandless

        Activity

        Hide
        mikemccand Michael McCandless added a comment -

        +1 to set limits and revert the getFiniteStrings change.

        Show
        mikemccand Michael McCandless added a comment - +1 to set limits and revert the getFiniteStrings change.
        Hide
        mikemccand Michael McCandless added a comment -

        Currently bad things happen if you send too-big an input to AnalyzingSuggester; here's my current patch that just adds a missing low-level check to the OfflineSorter so at least the user gets a clue that something was too big.

        If we enforce stricter limits higher up then we can leave the assert isFinite in place.

        Show
        mikemccand Michael McCandless added a comment - Currently bad things happen if you send too-big an input to AnalyzingSuggester; here's my current patch that just adds a missing low-level check to the OfflineSorter so at least the user gets a clue that something was too big. If we enforce stricter limits higher up then we can leave the assert isFinite in place.
        Hide
        rcmuir Robert Muir added a comment -

        Does this patch just catch too big at "index-time" ? Isn't it really a bugfix? Won't stuff just totally break without the check?

        Show
        rcmuir Robert Muir added a comment - Does this patch just catch too big at "index-time" ? Isn't it really a bugfix? Won't stuff just totally break without the check?
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1593548 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1593548 ]

        LUCENE-5660: add changs

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1593548 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1593548 ] LUCENE-5660 : add changs
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1593550 from Michael McCandless in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1593550 ]

        LUCENE-5660: add changs

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1593550 from Michael McCandless in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1593550 ] LUCENE-5660 : add changs
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1593551 from Michael McCandless in branch 'dev/branches/lucene_solr_4_8'
        [ https://svn.apache.org/r1593551 ]

        LUCENE-5660: add changes

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1593551 from Michael McCandless in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1593551 ] LUCENE-5660 : add changes
        Hide
        mikemccand Michael McCandless added a comment -

        OK I committed the bug-fix, so you see IAE with too-long-suggestion instead of scary corruption-like exceptions ...

        Show
        mikemccand Michael McCandless added a comment - OK I committed the bug-fix, so you see IAE with too-long-suggestion instead of scary corruption-like exceptions ...

          People

          • Assignee:
            Unassigned
            Reporter:
            rcmuir Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development