[TIKA-3864] Non-ascii UTF-8 characters in fetchKey not working with FileSystemFetcher

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.1
• Fix Version/s: 2.6.0
• Component/s: tika-pipes, tika-server

Environment

debian:bullseye docker container running tika-server-standard-2.4.1jar

• Assignee: Unassigned
• Reporter: Tong Wang
• Created: 27/Sep/22 20:56
• Updated: 03/Nov/22 17:40
• Resolved: 04/Oct/22 22:04

Description

When use FileSystemFetcher, if there is non-ascii characters in fetchKey, Tika Server throws exception because the file name is incorrect. Here is an example:

curl -v -X PUT http://tika:9998/rmeta/text --header "fetcherName: restricted" --header "fetchKey: 中文.txt" 

I get java.nio.file.NoSuchFileException:

Caused by: java.nio.file.NoSuchFileException: /restricted/ä¸æ–‡.txt	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)	at java.base/sun.nio.fs.UnixPath.toRealPath(UnixPath.java:860)	at org.apache.tika.pipes.fetcher.fs.FileSystemFetcher.fetch(FileSystemFetcher.java:64)	at org.apache.tika.server.core.FetcherStreamFactory.getInputStream(FetcherStreamFactory.java:90)	at org.apache.tika.server.core.resource.TikaResource.getInputStream(TikaResource.java:159) 

 

When I try to quote the characters:

curl -v -X PUT http://tika:9998/rmeta/text --header "fetcherName: restricted" --header "fetchKey: %E4%B8%AD%E6%96%87.txt" 

I still get a java.nio.file.NoSuchFileException:

Caused by: java.nio.file.NoSuchFileException: /restricted/%E4%B8%AD%E6%96%87.txt	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)	at java.base/sun.nio.fs.UnixPath.toRealPath(UnixPath.java:860)	at org.apache.tika.pipes.fetcher.fs.FileSystemFetcher.fetch(FileSystemFetcher.java:64)	at org.apache.tika.server.core.FetcherStreamFactory.getInputStream(FetcherStreamFactory.java:90)	at org.apache.tika.server.core.resource.TikaResource.getInputStream(TikaResource.java:159)

BTW, locale is set to C.UTF-8 on Tika Server:

# locale
LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=