[AIRFLOW-6347] Can't get task logs in webUI when DAG serialization is enabled

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.10.7
• Fix Version/s: 1.10.8
• Component/s: DAG, webserver
• Assignee: Kaxil Naik
• Reporter: Anton Cherkasov
• Created: 25/Dec/19 19:38
• Updated: 16/Jan/20 11:38
• Resolved: 07/Jan/20 18:29

Description

When I set next config options in airflow.cfg:

[core]
store_serialized_dags = True
min_serialized_dag_update_interval = 3

 

View with task logs shows infinity Js spinner, while in webserver log I see next error:

[2019-12-25 14:48:57,640] {{app.py:1891}} ERROR - Exception on /get_logs_with_metadata [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/www_rbac/decorators.py", line 121, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/security/decorators.py", line 101, in wraps
    return f(self, *args, **kwargs)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/www_rbac/decorators.py", line 56, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/www_rbac/views.py", line 637, in get_logs_with_metadata
    logs, metadata = _get_logs_with_metadata(try_number, metadata)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/www_rbac/views.py", line 628, in _get_logs_with_metadata
    logs, metadatas = handler.read(ti, try_number, metadata=metadata)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/utils/log/file_task_handler.py", line 169, in read
    log, metadata = self._read(task_instance, try_number_element, metadata)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/utils/log/file_task_handler.py", line 98, in _read
    log_relative_path = self._render_filename(ti, try_number)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/utils/log/file_task_handler.py", line 75, in _render_filename
    jinja_context = ti.get_template_context()
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1149, in get_template_context
    if 'tables' in task.params:
TypeError: argument of type 'NoneType' is not iterable

After some investigation I found that task.params field is empty dict when DAG serialization is disabled and this field became None object after enabling serialization.

Screenshot of Task Instance Details view with enabled DAG serialization feature:

 

Screenshot of Task Instance Details view with disabled DAG serialization feature:
 

 

See the params field.

Steps to reproduce:

1. Install Airflow 1.10.7 or upgrade from previous one
2. Upload some DAG and perform tasks.
3. Enable DAG serialization in airflow.conf
4. Restart airflow scheduler and webserver
5. Go to task's log view
6. See infinity spinner and errors in webserver logs

Issue Links

• is duplicated by: AIRFLOW-6578 store_serialized_dags raises fatal error for reviewing logs
• is related to: AIRFLOW-5088 To implement DAG JSON serialization and DB persistence for webserver scalability improvement
• links to: GitHub Pull Request #7092