[TIKA-2520] OptimaizeLangDetector#loadModels() should not be called for every single langdetect HTTP request

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.16
• Fix Version/s: 1.19
• Component/s: server
• Labels: performance
• Assignee: Chris A. Mattmann
• Reporter: Vincent van Donselaar
• Created: 08/Dec/17 15:08
• Updated: 26/May/18 01:29
• Resolved: 24/May/18 20:55

Description

Tika REST server's `/language` resource invokes the relatively heavy `loadModels` operation for every language detect call:

LanguageResource.java

public String detect(final String string) throws IOException {
	LanguageResult language = new OptimaizeLangDetector().loadModels().detect(string);
	String detectedLang = language.getLanguage();
	LOG.info("Detecting language for incoming resource: [{}]", detectedLang);
	return detectedLang;
}

This could be optimized by (lazy?) loading the models only once and keep them in memory. I assume the `LanguageDetector` is not thread safe, so I expect this requires an ExecutorService with language detectors.

Issue Links

• links to: GitHub Pull Request #237