[OAK-2895] Avoid accessing binary content if the mimeType is excluded from indexing

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.0.15, 1.2.3, 1.3.0, 1.4
• Component/s: lucene
• Labels: performance
• Assignee: Chetan Mehrotra
• Reporter: Chetan Mehrotra
• Created: 20/May/15 15:44
• Updated: 08/Oct/19 15:21
• Resolved: 21/May/15 10:51

Description

Currently the recommended way to exclude certain types of files from getting indexed is to add them to EmptyParser in Tika Config. However looking at how Tika works even if mimetype is provided as part metadata.

Tika Detector try to determine the mimetype by actually reading some bytes from InputStream [1] before looking up from passed MetaData. This would cause unnecessary IO in case large number of binaries are excluded.

We would need to look for way where any access to binary content which is not being indexed can be avoided. One option can to expose a multi value config property which takes a list of mimetypes to be excluded from indexing. If the mimeType provided as part of JCR data is part of that excluded list then call to Tika should be avoided

[1] https://github.com/apache/tika/blob/trunk/tika-core/src/main/java/org/apache/tika/mime/MimeTypes.java#L446

Issue Links

• is related to: OAK-5048 Upgrade to Tika 1.15 version