[LUCENE-1390] add ASCIIFoldingFilter and deprecate ISOLatin1AccentFilter

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.9
• Component/s: modules/analysis

Environment

any

• Lucene Fields: New, Patch Available
• Assignee: Mark Miller
• Reporter: Andi Vajda
• Created: 17/Sep/08 19:39
• Updated: 28/Aug/22 11:53
• Resolved: 06/Dec/08 23:28

Description

The ISOLatin1AccentFilter is removing accents from accented characters in the ISO Latin 1 character set.
It does what it does and there is no bug with it.

It would be nicer, though, if there was a more comprehensive version of this code that included not just ISO-Latin-1 (ISO-8859-1) but the entire Latin 1 and Latin Extended A unicode blocks.
See: http://en.wikipedia.org/wiki/Latin-1_Supplement_unicode_block
See: http://en.wikipedia.org/wiki/Latin_Extended-A_unicode_block

That way, all languages using roman characters are covered.
A new class, ISOLatinAccentFilter is attached. It is intended to supercede ISOLatin1AccentFilter which should get deprecated.

Issue Links

• relates to: LUCENE-1343 A replacement for AsciiFoldingFilter that does a more thorough job of removing diacritical marks or non-spacing modifiers.