[SOLR-9368] FileDictionaryFactory does not treat lines beginning with '#' as comments

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Not A Bug
• Affects Version/s: 6.1
• Component/s: Suggester
• Assignee: Unassigned
• Reporter: John Iacona
• Created: 02/Aug/16 09:02
• Updated: 08/Jun/19 15:38
• Resolved: 02/Aug/16 14:00

Description

The documentation for FileDictionaryFactory states that "Blank lines and lines that start with a '#' are ignored". This is not the case. When loading a dictionary file with '#' prefixed lines, they just get interpreted as terms and show up in suggestion results.

This causes additional confusion when trying to use payloads. As stated in https://lucene.apache.org/core/6_1_0/suggest/org/apache/lucene/search/suggest/FileDictionary.html : "In order to have payload enabled, the first entry has to have a payload". However, if you happen to have a "comment" as the first line in a dictionary file (that doesn't happen to have two instances of the fieldDelimiter in it...), payloads are disabled.