Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-6347

Can't get task logs in webUI when DAG serialization is enabled

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.10.7
    • Fix Version/s: 1.10.8
    • Component/s: DAG, webserver
    • Labels:
      None

      Description

      When I set next config options in airflow.cfg:

      [core]
      store_serialized_dags = True
      min_serialized_dag_update_interval = 3

       

      View with task logs shows infinity Js spinner, while in webserver log I see next error:

      [2019-12-25 14:48:57,640] {{app.py:1891}} ERROR - Exception on /get_logs_with_metadata [GET]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
          response = self.full_dispatch_request()
        File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
          rv = self.handle_user_exception(e)
        File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
          reraise(exc_type, exc_value, tb)
        File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
          raise value
        File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
          rv = self.dispatch_request()
        File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
          return self.view_functions[rule.endpoint](**req.view_args)
        File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/www_rbac/decorators.py", line 121, in wrapper
          return f(self, *args, **kwargs)
        File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/security/decorators.py", line 101, in wraps
          return f(self, *args, **kwargs)
        File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/www_rbac/decorators.py", line 56, in wrapper
          return f(*args, **kwargs)
        File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper
          return func(*args, **kwargs)
        File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/www_rbac/views.py", line 637, in get_logs_with_metadata
          logs, metadata = _get_logs_with_metadata(try_number, metadata)
        File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/www_rbac/views.py", line 628, in _get_logs_with_metadata
          logs, metadatas = handler.read(ti, try_number, metadata=metadata)
        File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/utils/log/file_task_handler.py", line 169, in read
          log, metadata = self._read(task_instance, try_number_element, metadata)
        File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/utils/log/file_task_handler.py", line 98, in _read
          log_relative_path = self._render_filename(ti, try_number)
        File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/utils/log/file_task_handler.py", line 75, in _render_filename
          jinja_context = ti.get_template_context()
        File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper
          return func(*args, **kwargs)
        File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1149, in get_template_context
          if 'tables' in task.params:
      TypeError: argument of type 'NoneType' is not iterable

      After some investigation I found that task.params field is empty dict when DAG serialization is disabled and this field became None object after enabling serialization.

      Screenshot of Task Instance Details view with enabled DAG serialization feature:

       

      Screenshot of Task Instance Details view with disabled DAG serialization feature:
       

       

      See the params field.

      Steps to reproduce:

      1. Install Airflow 1.10.7 or upgrade from previous one
      2. Upload some DAG and perform tasks.
      3. Enable DAG serialization in airflow.conf
      4. Restart airflow scheduler and webserver
      5. Go to task's log view
      6. See infinity spinner and errors in webserver logs

        Attachments

        1. serialization-disabled.png
          62 kB
          Anton Cherkasov
        2. serialization_enabled.png
          61 kB
          Anton Cherkasov

          Issue Links

            Activity

              People

              • Assignee:
                kaxilnaik Kaxil Naik
                Reporter:
                acherkasov Anton Cherkasov
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: