Uploaded image for project: 'Commons VFS'
  1. Commons VFS
  2. VFS-555

Add ability to specify alternate HDFS configuration file

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1
    • Fix Version/s: 2.1
    • Labels:
      None
    • Environment:

      All

      Description

      We need to access HDFS on an AWS installation from outside the AWS firewall. The technique is described here: http://rainerpeter.wordpress.com/2014/02/12/connect-to-hdfs-running-in-ec2-using-public-ip-addresses/
      In order to do this we need to set the "dfs.client.use.datanode.hostname=true" in an alternate configuration file set after the default configuration is loaded.
      I propose to add a property "configName" to HdfsFileSystemConfigBuilder, and then code in HdfsFileSystem to get the value of the property and set it in the Configuration.

        Attachments

        1. VFS-555.patch
          3 kB
          Roger Whitcomb

          Issue Links

            Activity

              People

              • Assignee:
                b.eckenfels Bernd Eckenfels
                Reporter:
                rwhitcomb Roger Whitcomb
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: