Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16288

Using an authenticated proxy server to access ADLS and ABFS storage

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.2.0
    • None
    • fs/adl, fs/azure
    • None

    Description

      Given an environment which does not have direct access to the internet, but has to route all requests through a Proxy that requires authentication Hadoop filesystem commands can not go through.

      My understanding of an Official Java blog, is that  this requires a special Authenticator class to be used, which I do not see anywhere in the code, neither I could find any relevant parameter to set the proxy authentication credentials.

      I tried to specify the proxy in a form of username:password@host.fqdn and the port via java system properties, but that did not work either.

      My use case is to connect to ADLS, but I have not seen relevant properties either for S3.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            pifta István Fajth

            Dates

              Created:
              Updated:

              Slack

                Issue deployment