Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-3397

Adding a new role to rbac makes airflow crash

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 1.10.2
    • Component/s: security
    • Labels:
      None

      Description

      Here are the steps to reproduce:

      • Start airflow with rbac enabled
      • Create a new role
      • Wait until one of the gunicorn workers is restarted
      • Airflow crashes with the following exception:

      ```
      [2018-11-26 09:59:17,353] {_init_.py:51} INFO - Using executor KubernetesExecutor
      [2018-11-26 09:59:17,617] {models.py:286} INFO - Filling up the DagBag from /root/airflow/dags
      [2018-11-26 09:59:18,134] {security.py:435} INFO - Start syncing user roles.
      [2018-11-26 09:59:18,270] {security.py:194} INFO - Existing permissions for the role:Viewer within the database will persist.
      /usr/local/lib/python2.7/dist-packages/flask_appbuilder/fields.py:152: UserWarning: allow_blank=True does not do anything for QuerySelectMultipleField.
        warnings.warn('allow_blank=True does not do anything for QuerySelectMultipleField.')
      [2018-11-26 09:59:18,384] {security.py:194} INFO - Existing permissions for the role:User within the database will persist.
      [2018-11-26 09:59:18,491] {security.py:194} INFO - Existing permissions for the role:Op within the database will persist.
      [2018-11-26 09:59:18,491] {security.py:344} INFO - Fetching a set of all permission, view_menu from FAB meta-table
      172.17.0.1 - - [26/Nov/2018:09:59:18 +0000] "GET /roles/list/ HTTP/1.1" 200 48607 "http://localhost/home" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0"
      [2018-11-26 09:59:18 +0000] [37] [ERROR] Exception in worker process
      Traceback (most recent call last):
        File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 583, in spawn_worker
          worker.init_process()
        File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 129, in init_process
          self.load_wsgi()
        File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 138, in load_wsgi
          self.wsgi = self.app.wsgi()
        File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
          self.callable = self.load()
        File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load
          return self.load_wsgiapp()
        File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
          return util.import_app(self.app_uri)
        File "/usr/local/lib/python2.7/dist-packages/gunicorn/util.py", line 362, in import_app
          app = eval(obj, vars(mod))
        File "<string>", line 1, in <module>
        File "/usr/local/lib/python2.7/dist-packages/airflow/www_rbac/app.py", line 207, in cached_app
          app, _ = create_app(config, session, testing)
        File "/usr/local/lib/python2.7/dist-packages/airflow/www_rbac/app.py", line 167, in create_app
          security_manager.sync_roles()
        File "/usr/local/lib/python2.7/dist-packages/airflow/www_rbac/security.py", line 443, in sync_roles
          self.create_custom_dag_permission_view()
        File "/usr/local/lib/python2.7/dist-packages/airflow/www_rbac/security.py", line 406, in create_custom_dag_permission_view
          self.get_session.execute(ab_perm_view_role.insert(), update_perm_views)
        File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/scoping.py", line 157, in do
          return getattr(self.registry(), name)(*args, **kwargs)
        File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1160, in execute
          bind, close_with_result=True).execute(clause, params or {})
        File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
          return meth(self, multiparams, params)
        File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
          return connection._execute_clauseelement(self, multiparams, params)
        File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
          compiled_sql, distilled_params
        File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
          context)
        File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
          exc_info
        File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
          reraise(type(exception), exception, tb=exc_tb, cause=cause)
        File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1159, in _execute_context
          context)
        File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 467, in do_executemany
          cursor.executemany(statement, parameters)
      IntegrityError: (psycopg2.IntegrityError) insert or update on table "ab_permission_view_role" violates foreign key constraint "ab_permission_view_role_role_id_fkey"
      DETAIL:  Key (role_id)=(347) is not present in table "ab_role".
       [SQL: "INSERT INTO ab_permission_view_role (id, permission_view_id, role_id) VALUES (nextval('ab_permission_view_role_id_seq'), %(permission_view_id)s, %(role_id)s)"] [parameters: (\{'permission_view_id': 36, 'role_id': 347}, \{'permission_view_id': 37, 'role_id': 347}, \{'permission_view_id': 38, 'role_id': 347}, \{'permission_view_id': 39, 'role_id': 347}, \{'permission_view_id': 40, 'role_id': 347}, \{'permission_view_id': 41, 'role_id': 347}, \{'permission_view_id': 42, 'role_id': 347}, \{'permission_view_id': 43, 'role_id': 347}  ... displaying 10 of 54 total bound parameter sets ...  \{'permission_view_id': 122, 'role_id': 347}, \{'permission_view_id': 123, 'role_id': 347})]
      [2018-11-26 09:59:18 +0000] [37] [INFO] Worker exiting (pid: 37)
      [2018-11-26 09:59:19 +0000] [33] [INFO] Worker exiting (pid: 33)
      [2018-11-26 09:59:19 +0000] [15] [INFO] Worker exiting (pid: 15)
      [2018-11-26 09:59:19 +0000] [16] [INFO] Worker exiting (pid: 16)
      [2018-11-26 09:59:19 +0000] [29] [INFO] Worker exiting (pid: 29)
      [2018-11-26 09:59:19 +0000] [8] [INFO] Shutting down: Master
      [2018-11-26 09:59:19 +0000] [8] [INFO] Reason: Worker failed to boot.
      [2018-11-26 09:59:19,963] {cli.py:818} ERROR - [0 / 0] some workers seem to have died and gunicorndid not restart them as expected
      ```

      Solution:
      This is caused because of name shadowing in a for loop.
      The following diff solves the problem:
      ```
      diff --git a/airflow/www_rbac/security.py b/airflow/www_rbac/security.py
      index 8f9b6287..369ec71c 100644
      — a/airflow/www_rbac/security.py
      +++ b/airflow/www_rbac/security.py
      @@ -395,8 +395,8 @@ class AirflowSecurityManager(SecurityManager):
                   existing_perm_view_by_user = self.get_session.query(ab_perm_view_role)\
                       .filter(ab_perm_view_role.columns.role_id == role.id)
       
      -            existing_perms_views = set([role.permission_view_id
      -                                        for role in existing_perm_view_by_user])
      +            existing_perms_views = set([pv.permission_view_id
      +                                        for pv in existing_perm_view_by_user])
                   missing_perm_views = all_perm_views - existing_perms_views
       
                   for perm_view_id in missing_perm_views:
      ```

      I have no time atm to create a PR, but I can do it later if required.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                BrechtDeVlieger Brecht De Vlieger
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: