[TIKA-1549] Two times speed increase of language profile distance calculation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.8
• Component/s: languageidentifier
• Assignee: Chris A. Mattmann
• Reporter: Toke Eskildsen
• Created: 14/Feb/15 07:19
• Updated: 14/Feb/15 14:11
• Resolved: 14/Feb/15 07:50

Description

The distance calculation for language profiles creates a Set of Strings with each call and performs hashtable lookups for all those Strings. This patch creates and caches compact structures that are iterated sequentially. The result is a factor 2 speed up (see the performance test in LanguageIdentifierTest).