[FLINK-29754] HadoopConfigLoader should consider Hadoop configuration files - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: FileSystems
Labels:
None

Description

Currently HadoopConfigLoader considers Hadoop configurations on the classpath, but does not consider Hadoop configuration files which are set in another way.

So if the Hadoop configuration is set through the HADOOP_CONF_DIR environment variable, then the configuration loaded by the HadoopConfigLoader will not contain the values set there.

This can cause unexpected behaviour when setting checkpoint / savepoint dirs on S3, and the specific S3 configurations are set in the Hadoop configuration files

Attachments

Activity

People

Assignee:: Unassigned

Reporter:: Peter Vary

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 25/Oct/22 08:21

Updated:: 25/Oct/22 14:48