[TIKA-3137] Enable a metadata filter for the RecursiveParserWrapper

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.25
• Assignee: Tim Allison
• Reporter: Tim Allison
• Created: 16/Jul/20 15:04
• Updated: 17/Jul/20 19:26
• Resolved: 17/Jul/20 17:19

Description

The RecursiveParserWrapper is designed to extract all metadata from every embedded file. Some users may need more targeted ways of filtering the metadata to save on resources, e.g. memory, disc or transfer-size/bandwidth in tika-server.

Some use cases that come to mind:

• A user only wants the title, author and content fields.
• A user doesn't want content from EMF files, but does want the content from a PDF embedded inside an EMF file.
• This could be an avenue for text-based enrichment, e.g. run NER on the content field and add those recognized entities to the Metadata; or tika-eval statistics...

The last point may require further discussion. We have some handlers that require buffering the full text of a document and then running extraction (Phone number extractor?). The downside to this is that we're storing two copies of the data in memory. For at least the RPW, it would be more efficient to do postprocessing on the one buffered copy.

Some open questions: how do we configure the choice of filter(s), do we apply this to the AutoDetectParser...