[FLINK-17359] Entropy key is not resolved if flink-s3-fs-hadoop is added as a plugin

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.10.1, 1.11.0
• Component/s: FileSystems
• Labels: pull-request-available
• Assignee: Prem Santosh
• Reporter: Prem Santosh
• Created: 23/Apr/20 21:39
• Updated: 01/Apr/21 04:48
• Resolved: 27/Apr/20 08:25

Description

Using flink 1.10

I added the flink-s3-fs-hadoop jar in plugins dir but I am seeing the checkpoints paths like s3://my_app/_ENTROPY_/app_name-staging/flink/checkpoints/e10f47968ae74934bd833108d2272419/chk-3071 which means the entropy injection key is not being resolved. After some debugging I found that in the EntropyInjector we check if the given fileSystem is of type SafetyNetWrapperFileSystem and if so we check if the delegate is of type EntropyInjectingFileSystem but don't check for ClassLoaderFixingFileSystem which would be the type if S3 file system dependencies are added as a plugin.

Issue Links

• is cloned by: FLINK-22081 Entropy key not resolved if flink-s3-fs-hadoop is added as a plugin
• links to: GitHub Pull Request #11891
• links to: GitHub Pull Request #15442