Uploaded image for project: 'Tika'
  1. Tika
  2. TIKA-3137

Enable a metadata filter for the RecursiveParserWrapper

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.25
    • None
    • None

    Description

      The RecursiveParserWrapper is designed to extract all metadata from every embedded file. Some users may need more targeted ways of filtering the metadata to save on resources, e.g. memory, disc or transfer-size/bandwidth in tika-server.

      Some use cases that come to mind:

      • A user only wants the title, author and content fields.
      • A user doesn't want content from EMF files, but does want the content from a PDF embedded inside an EMF file.
      • This could be an avenue for text-based enrichment, e.g. run NER on the content field and add those recognized entities to the Metadata; or tika-eval statistics...

      The last point may require further discussion. We have some handlers that require buffering the full text of a document and then running extraction (Phone number extractor?). The downside to this is that we're storing two copies of the data in memory. For at least the RPW, it would be more efficient to do postprocessing on the one buffered copy.

      Some open questions: how do we configure the choice of filter(s), do we apply this to the AutoDetectParser...

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            tallison Tim Allison
            tallison Tim Allison
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment