[CONNECTORS-1700] TikaServiceRmeta: Add options to filter out metadata based on size

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: ManifoldCF 2.21
• Fix Version/s: ManifoldCF 2.22
• Component/s: Tika service connector
• Assignee: Julien Massiera
• Reporter: Julien Massiera
• Created: 15/Mar/22 16:28
• Updated: 30/Apr/22 15:39
• Resolved: 15/Mar/22 16:34

Description

Some files may contain abnormally big metadata (several MB, be it for the metadata values, but also for the total amount of metadata) that can be problematic concerning the memory consumption of the connector. 

To avoid this, we can provide job configuration options for the TikaServiceRmetaConnector to set limits on both metadata values and global amount of metadata, and exclude metadata that exceed the limits