[SOLR-2450] Carrot2 clustering should use both its own and Solr's stop words

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.2, 4.0-ALPHA
• Component/s: contrib - Clustering
• Assignee: Stanislaw Osinski
• Reporter: Stanislaw Osinski
• Created: 30/Mar/11 17:32
• Updated: 02/May/13 02:29
• Resolved: 16/May/11 16:30

Description

While using only Solr's stop words for clustering isn't a good idea (compared to indexing, clustering needs more aggressive stop word removal to get reasonable cluster labels), it would be good if Carrot2 used both its own and Solr's stop words.

I'm not sure what the best way to implement this would be though. My first thought was to simply load stopwords.txt from Solr config dir and merge them with Carrot2's. But then, maybe a better approach would be to get the stop words from the StopFilter being used? Ideally, we should also consider the per-field stop filters configured on the fields used for clustering.

Issue Links

• depends upon: SOLR-2448 Upgrade Carrot2 to version 3.5.0