Lucene - Core
  1. Lucene - Core
  2. LUCENE-5660

AnalyzingSuggester needs reasonable limits on max suggestion length

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Issues like LUCENE-5628 LUCENE-5659 are hacks around the problem that the AnalyzingSuggester allows automata to "explode".

      I don't think we should try to hack up the automata lib to the point its un-understandable to support this, since its designed for smaller automata such as queries and is appropriate for that.

      We should instead just set appropriate limits out of box so AnalyzingSuggester doesnt blow up.

      Remember this is a suggester, it needs to be fast. The fact that the automata lib blows up on stupid unit tests or whatever are doing this, instead of being silently slow, is a good thing.

      1. LUCENE-5660.patch
        3 kB
        Michael McCandless

        Activity

        Hide
        Michael McCandless added a comment -

        +1 to set limits and revert the getFiniteStrings change.

        Show
        Michael McCandless added a comment - +1 to set limits and revert the getFiniteStrings change.
        Hide
        Michael McCandless added a comment -

        Currently bad things happen if you send too-big an input to AnalyzingSuggester; here's my current patch that just adds a missing low-level check to the OfflineSorter so at least the user gets a clue that something was too big.

        If we enforce stricter limits higher up then we can leave the assert isFinite in place.

        Show
        Michael McCandless added a comment - Currently bad things happen if you send too-big an input to AnalyzingSuggester; here's my current patch that just adds a missing low-level check to the OfflineSorter so at least the user gets a clue that something was too big. If we enforce stricter limits higher up then we can leave the assert isFinite in place.
        Hide
        Robert Muir added a comment -

        Does this patch just catch too big at "index-time" ? Isn't it really a bugfix? Won't stuff just totally break without the check?

        Show
        Robert Muir added a comment - Does this patch just catch too big at "index-time" ? Isn't it really a bugfix? Won't stuff just totally break without the check?
        Hide
        ASF subversion and git services added a comment -

        Commit 1593548 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1593548 ]

        LUCENE-5660: add changs

        Show
        ASF subversion and git services added a comment - Commit 1593548 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1593548 ] LUCENE-5660 : add changs
        Hide
        ASF subversion and git services added a comment -

        Commit 1593550 from Michael McCandless in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1593550 ]

        LUCENE-5660: add changs

        Show
        ASF subversion and git services added a comment - Commit 1593550 from Michael McCandless in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1593550 ] LUCENE-5660 : add changs
        Hide
        ASF subversion and git services added a comment -

        Commit 1593551 from Michael McCandless in branch 'dev/branches/lucene_solr_4_8'
        [ https://svn.apache.org/r1593551 ]

        LUCENE-5660: add changes

        Show
        ASF subversion and git services added a comment - Commit 1593551 from Michael McCandless in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1593551 ] LUCENE-5660 : add changes
        Hide
        Michael McCandless added a comment -

        OK I committed the bug-fix, so you see IAE with too-long-suggestion instead of scary corruption-like exceptions ...

        Show
        Michael McCandless added a comment - OK I committed the bug-fix, so you see IAE with too-long-suggestion instead of scary corruption-like exceptions ...

          People

          • Assignee:
            Unassigned
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development