[AIRFLOW-2703] Scheduler crashes if Mysql Connectivity is lost - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 1.9.0, 2.0.0
Fix Version/s: 1.10.2
Component/s: scheduler
Labels:
None

Description

Airflow scheduler crashes if connectivity to Mysql is lost.

Below is the stack Trace
Traceback (most recent call last): File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 371, in helper pickle_dags) File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper result = func(*args, **kwargs) File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 1762, in process_file dag.sync_to_db() File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper result = func(*args, **kwargs) File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/models.py", line 3816, in sync_to_db session.commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 943, in commit self.transaction.commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 471, in commit t[1].commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1643, in commit self._do_commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1674, in _do_commit self.connection._commit_impl() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 726, in _commit_impl self._handle_dbapi_exception(e, None, None, None, None) File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception exc_info File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 724, in _commit_impl self.engine.dialect.do_commit(self.connection) File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/mysql/base.py", line 1784, in do_commit dbapi_connection.commit() OperationalError: (_mysql_exceptions.OperationalError) (2013, 'Lost connection to MySQL server during query') (Background on this error at: http://sqlalche.me/e/e3q8)
Process DagFileProcessor141318-Process: