[FLINK-7643] Configure FileSystems only once

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.0
• Fix Version/s: 1.4.0
• Component/s: Runtime / State Backends
• Assignee: Stephan Ewen
• Reporter: Ufuk Celebi
• Created: 19/Sep/17 13:59
• Updated: 08/Nov/17 13:18
• Resolved: 08/Nov/17 13:15

Description

HadoopFileSystem always reloads GlobalConfiguration, which potentially leads to a lot of noise in the logs, because this happens on each checkpoint.

Instead, file systems should be configured once upon process startup, when the configuration is loaded.

This will also increase efficiency of checkpoints, as it avoids redundant parsing for each data chunk.

Issue Links

• is duplicated by: FLINK-7365 excessive warning logs of attempt to override final parameter: fs.s3.buffer.dir
• links to: GitHub Pull Request #4776