Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.0.1, 5.5.2, 5.6, 7.0
    • Component/s: modules/analysis
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      On certain Japanese input strings you can hit this:

      java.lang.ArrayIndexOutOfBoundsException: -1
      	at __randomizedtesting.SeedInfo.seed([C6752A567B924B1:2B195610610ED60]:0)
      	at org.apache.lucene.analysis.ja.JapaneseTokenizer.backtrace(JapaneseTokenizer.java:1607)
      	at org.apache.lucene.analysis.ja.JapaneseTokenizer.parse(JapaneseTokenizer.java:902)
      	at org.apache.lucene.analysis.ja.JapaneseTokenizer.incrementToken(JapaneseTokenizer.java:479)
      	at org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testBigDocument(TestJapaneseTokenizer.java:837)
      

      I have a patch with a test case and fix.

      1. LUCENE-7279.patch
        10 kB
        Michael McCandless

        Activity

        Hide
        mikemccand Michael McCandless added a comment -

        The bug is that unknownWordIndex needs to be reset per-parse and not per- input string.

        Show
        mikemccand Michael McCandless added a comment - The bug is that unknownWordIndex needs to be reset per-parse and not per- input string.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit c1d5a5eb3581d554dbfce08b0241c53c9f58ad61 in lucene-solr's branch refs/heads/branch_6_0 from Mike McCandless
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c1d5a5e ]

        LUCENE-7279: don't throw AIOOBE on some valid inputs

        Show
        jira-bot ASF subversion and git services added a comment - Commit c1d5a5eb3581d554dbfce08b0241c53c9f58ad61 in lucene-solr's branch refs/heads/branch_6_0 from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c1d5a5e ] LUCENE-7279 : don't throw AIOOBE on some valid inputs
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 145eaf872b105c7facf30bf29e415a070d0a5946 in lucene-solr's branch refs/heads/branch_6_0 from Mike McCandless
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=145eaf8 ]

        LUCENE-7279: add CHANGES

        Show
        jira-bot ASF subversion and git services added a comment - Commit 145eaf872b105c7facf30bf29e415a070d0a5946 in lucene-solr's branch refs/heads/branch_6_0 from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=145eaf8 ] LUCENE-7279 : add CHANGES
        Hide
        mikemccand Michael McCandless added a comment -

        I backported this for 6.0.1.

        Show
        mikemccand Michael McCandless added a comment - I backported this for 6.0.1.
        Hide
        steve_rowe Steve Rowe added a comment -

        Bulk close issues included in the 6.0.1 release.

        Show
        steve_rowe Steve Rowe added a comment - Bulk close issues included in the 6.0.1 release.
        Hide
        steve_rowe Steve Rowe added a comment -

        Reopening to backport to 5.6 and 5.5.2

        Show
        steve_rowe Steve Rowe added a comment - Reopening to backport to 5.6 and 5.5.2
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 4a824d62e280f10ad58b43b20d6fe593cabcfd00 in lucene-solr's branch refs/heads/branch_5_5 from Mike McCandless
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4a824d6 ]

        LUCENE-7279: don't throw AIOOBE on some valid inputs

        Show
        jira-bot ASF subversion and git services added a comment - Commit 4a824d62e280f10ad58b43b20d6fe593cabcfd00 in lucene-solr's branch refs/heads/branch_5_5 from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4a824d6 ] LUCENE-7279 : don't throw AIOOBE on some valid inputs
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 2a3492574b470ca49666f53b66ffa6394a9a78d2 in lucene-solr's branch refs/heads/branch_5_5 from Steve Rowe
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2a34925 ]

        LUCENE-7279: add CHANGES entry

        Show
        jira-bot ASF subversion and git services added a comment - Commit 2a3492574b470ca49666f53b66ffa6394a9a78d2 in lucene-solr's branch refs/heads/branch_5_5 from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2a34925 ] LUCENE-7279 : add CHANGES entry
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit bcf1eb7d24810eae7123c89e079823ce56b9dd25 in lucene-solr's branch refs/heads/branch_5x from Mike McCandless
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bcf1eb7 ]

        LUCENE-7279: don't throw AIOOBE on some valid inputs

        Show
        jira-bot ASF subversion and git services added a comment - Commit bcf1eb7d24810eae7123c89e079823ce56b9dd25 in lucene-solr's branch refs/heads/branch_5x from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bcf1eb7 ] LUCENE-7279 : don't throw AIOOBE on some valid inputs
        Hide
        steve_rowe Steve Rowe added a comment -

        Bulk close issues released with 5.5.2.

        Show
        steve_rowe Steve Rowe added a comment - Bulk close issues released with 5.5.2.

          People

          • Assignee:
            mikemccand Michael McCandless
            Reporter:
            mikemccand Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development