Uploaded image for project: 'Tika'
  1. Tika
  2. TIKA-3137

Enable a metadata filter for the RecursiveParserWrapper

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.25
    • Component/s: None
    • Labels:
      None

      Description

      The RecursiveParserWrapper is designed to extract all metadata from every embedded file. Some users may need more targeted ways of filtering the metadata to save on resources, e.g. memory, disc or transfer-size/bandwidth in tika-server.

      Some use cases that come to mind:

      • A user only wants the title, author and content fields.
      • A user doesn't want content from EMF files, but does want the content from a PDF embedded inside an EMF file.
      • This could be an avenue for text-based enrichment, e.g. run NER on the content field and add those recognized entities to the Metadata; or tika-eval statistics...

      The last point may require further discussion. We have some handlers that require buffering the full text of a document and then running extraction (Phone number extractor?). The downside to this is that we're storing two copies of the data in memory. For at least the RPW, it would be more efficient to do postprocessing on the one buffered copy.

      Some open questions: how do we configure the choice of filter(s), do we apply this to the AutoDetectParser...

        Attachments

          Activity

            People

            • Assignee:
              tallison Tim Allison
              Reporter:
              tallison Tim Allison
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: