Uploaded image for project: 'Commons VFS'
  1. Commons VFS
  2. VFS-586

Add more ways to specify alternate HDFS configuration information

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.1
    • Fix Version/s: None
    • Labels:
      None
    • Environment:

      All

      Description

      In trying to resolve some customer issues we were experiencing, it was necessary to specify some alternate HDFS configuration information from some places other than resources in the current classpath, plus needing to specify multiple configurations (such as a local copy of "core-site.xml" and "hdfs-site.xml"). To accomplish this I have proposed some changes to HdfsFileSystemConfigBuilder.java and HdfsFileSystem.java to be able to specify alternate configurations from all the possible sources (as defined in org.apache.hadoop.fs.Configuration), namely from a named resource, from a local file Path, from an InputStream and from another Configuration object. For resource names and local files, multiple entries are allowed so that multiple configuration files can be specified as needed.

        Attachments

        1. hdfs-config-2.diffs
          15 kB
          Roger Lee Whitcomb

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                rwhitcomb Roger Lee Whitcomb
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: