[LUCENE-6914] DecimalDigitFilter skips characters in some cases (supplemental?)

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.4
• Fix Version/s: 5.5.4, 6.3, 7.0
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Chris M. Hostetter
• Created: 30/Nov/15 21:54
• Updated: 28/Aug/22 14:46
• Resolved: 08/Feb/17 10:26

Description

Found this while writing up the solr ref guide for DecimalDigitFilter.

With input like "𝟙𝟡𝟠𝟜" ("Double Struck" 1984) the filter produces "1𝟡8𝟜" (1, double struck 9, 8, double struck 4) add some non-decimal characters in between the digits (ie: "𝟙x𝟡x𝟠x𝟜") and you get the expected output ("1x9x8x4"). This doesn't affect all decimal characters though, as evident by the existing test cases.

Perhaps this is an off by one bug in the "if the original was supplementary, shrink the string" code path?

Issue Links

• is duplicated by: LUCENE-7363 DecimalDigitFilter skips chars in case of supplementary code points
• is related to: LUCENE-6737 Add DecimalDigitFilter