[FLINK-9525] Missing META-INF/services/*FileSystemFactory in flink-hadoop-fs module

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.4.0, 1.5.0, 1.6.0
• Component/s: FileSystems
• Labels: pull-request-available
• Assignee: Hai Zhou
• Reporter: Hai Zhou
• Created: 05/Jun/18 06:34
• Updated: 07/Feb/19 10:45
• Resolved: 07/Feb/19 10:45

Description

if flink job dependencies includes `hadoop-common` and `hadoop-hdfs`, will throw runtime error.
like this case: https://stackoverflow.com/questions/47890596/java-util-serviceconfigurationerror-org-apache-hadoop-fs-filesystem-provider-o.

the root cause:
see org.apache.flink.core.fs.FileSystem
This class will load all available file system factories via ServiceLoader.load(FileSystemFactory.class).
Since {{ META-INF / services / org.apache.flink.core.fs.FileSystemFactory }} file in the classpath does not have an `org.apache.flink.runtime.fs.hdfs.HadoopFsFactory`,
and finaly only loaded one available LocalFileSystemFactory .

more error messages see this screenshot.

Issue Links

• links to: GitHub Pull Request #6118
• links to: GitHub Pull Request #6118