[VFS-586] Add more ways to specify alternate HDFS configuration information

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 2.1

Environment

All

• Assignee: Unassigned
• Reporter: Roger Lee Whitcomb
• Created: 17/Nov/15 19:16
• Updated: 31/Oct/20 20:08

Description

In trying to resolve some customer issues we were experiencing, it was necessary to specify some alternate HDFS configuration information from some places other than resources in the current classpath, plus needing to specify multiple configurations (such as a local copy of "core-site.xml" and "hdfs-site.xml"). To accomplish this I have proposed some changes to HdfsFileSystemConfigBuilder.java and HdfsFileSystem.java to be able to specify alternate configurations from all the possible sources (as defined in org.apache.hadoop.fs.Configuration), namely from a named resource, from a local file Path, from an InputStream and from another Configuration object. For resource names and local files, multiple entries are allowed so that multiple configuration files can be specified as needed.

Issue Links

• relates to: VFS-555 Add ability to specify alternate HDFS configuration file