Affects Version/s: 1.10.9
Fix Version/s: None
I have noticed this problem with SimpleHttpOperator in Airflow 1.10.9.
When we set it up with conn_id, it does not take into account 'extra' parameters from connection.
Airflow -> Admin -> Connection -> Extra:
My python skill is very bad, but I think there are 2 things happening:
http_hook.py - function get_conn - line 78
This is updating session header, however header can't accept type dictionary, only string. So first error with this is that the header is incorrectly formatted (type error).
To me, it makes no sense to use extra field as header source, but if this have to be kept for compatibility reason, I would suggest excluding python-request parameters from pushing to headers. Otherwise It may results also in insecure behavior.
*http_hook.py - function *run_and_check - ** lines 176-184
Extra options here are only taken into consideration from extra_option object, but they they should be taken from session object as well and overwritten by connection extra_option object afterwards.
PS: This prohibits us from having sensitive information like proxy login in secure place therefore creates security concern.