[TIKA-4246] tika-pipes FileSystemFetcher configuration option for file name/path pattern selection

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: tika-pipes
• Assignee: Unassigned
• Reporter: Emil Zegers
• Created: 26/Apr/24 13:18
• Updated: 26/Apr/24 13:59

Description

Would be useful to have the possibility to configure FileSystemFetcher for tika-pipes to only process certain files, e.g. based on extension, match on file name/path or similar pattern.
 
This way it would be possible to point to a specific root folder and only process matching files like certain extensions, names (e.g. for GIS files like shapefiles there is same name with multiple extensions) etc.
 
Something like:
 

<properties>
  <fetchers>
    <fetcher class="org.apache.tika.pipes.fetcher.fs.FileSystemFetcher">
      <params>
        <name>fsf</name>
        <basePath>/my/base/path1</basePath>
        <pattern>myshapefilename.*</pattern>
      </params>
    </fetcher>
  </fetchers>
</properties>

 
Or:
 

        <pattern>*.doc*,*.pdf</pattern>