Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-3264

Only '/' is URL-decoded when parsing hostname in the Connection

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.10.0
    • Fix Version/s: 1.10.1, 1.10.2
    • Component/s: database
    • Labels:
      None

      Description

      Parsing of the URL for connection(using AIRFLOW_CONN_ environment variables) does not perform full URL-decode on the URL. It only handles hard-coded %2f encode to support "/" in hostname. However there are valid cases where the hostname, login, password, and query parameters can contain url-encoded values. For example in cloud-sql-proxy, generated socket path contains ":" (for example /cloudsql/myProject:us-central1:myInstance)

      We need to URL-encode ":" because otherwise urlparse will treat those ":" as separator for port number. Similarly user/password can contain url-encoded characters.

      I think we should fully URL-decode all relevant URL fields (including query parameters, user, password, hostname, path). However it is potentially breaking change (if someone has a user/password/hostname with % ) so maybe we should do some compromises around that (for example not decode the password - which are likely to contain '%' characters) although that would violate URL encoding/decoding specification.

      I will provide proposed fix shortly

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                potiuk Jarek Potiuk
                Reporter:
                potiuk Jarek Potiuk
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: