Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-2759

Simplify proxy server based access to external platforms like Google cloud

Agile BoardAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersLinkUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: hooks
    • Labels:

      Description

      Several companies adopt a Proxy Server based approach in order to provide an additional layer of security while communicating with external platforms to establish legitimacy of caller and calle. A potential use case would be writing logs from Airflow to a cloud storage platform like google cloud via an intermediary proxy server.

      In the current scenario the proxy details need to be hardcoded and passed to the HTTP client library(httplib2) in the GoogleCloudBaseHook class (gcp_api_base_hook.py). It would be convenient if the proxy details (for example, host and port) can be extracted from the airflow configuration file as opposed to hardcoding the details at hook level.

        Attachments

          Activity

            People

            • Assignee:
              aimohan Aishwarya Mohan Assign to me
              Reporter:
              aimohan Aishwarya Mohan

              Dates

              • Created:
                Updated:

                Issue deployment