Lucene - Core
  1. Lucene - Core
  2. LUCENE-5824

hunspell FLAG LONG implemented incorrectly

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.10, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      If you have more than 256 flags, you run out of 8-bit characters, so you have to use another flag type to get 64k:

      • UTF-8: 16-bit BMP flags
      • long: two-character flags like 'AB'
      • num: decimal numbers like '10234'

      But our implementation for 'long' is wrong, it encodes as 'A+B', which means it cant distinguish between 'AB' and 'BA' and causes overgeneration.

        Activity

        Hide
        Robert Muir added a comment -

        Simple patch and test to encode as A << 8 + B (and also check the values are really within range: they should be two ascii characters).

        This bug currently impacts the more complicated dictionaries using this encoding type (russian, arabic, hebrew, etc)

        Show
        Robert Muir added a comment - Simple patch and test to encode as A << 8 + B (and also check the values are really within range: they should be two ascii characters). This bug currently impacts the more complicated dictionaries using this encoding type (russian, arabic, hebrew, etc)

          People

          • Assignee:
            Unassigned
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development