Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-2316

get_hook not updated for many connection types - can result in silent failure for BaseHook.get_hook()

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.8.0, 1.9.0, 1.10.0
    • Fix Version/s: None
    • Component/s: models
    • Labels:
      None

      Description

      Currently adding a new hook requires adjusting the Connection.get_hook() method in models.py. This seems to be overlooked by many hooks. Which in turn results in a (silent) failure to load hooks based on connection id only.

      e.g. 

      executing BaseHook.get_hook('s3connection') will return None because there is no

       elif self.conn_type == 's3':
      

      statement in Connection.get_hook. Currently statements seem to be missing for the following connection types: fs, ftp, hdfs, http, hive_metastore, s3, samba, ssh, mesos_framework-id, databricks, aws, emr. 

      Possible fix is to just add the extra elif statements for all of the missing connection types. To prevent this in the future a unit test can be added that checks the get_hook method returns an object for all items in the Connection._types list.  

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Bouke Bouke Nederstigt
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: