[AIRFLOW-2703] Scheduler crashes if Mysql Connectivity is lost - ASF JIRA

Agile Board

Attach files

Attach Screenshot

Voters

Stop watching

Watchers

Link

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 1.9.0, 2.0.0
Fix Version/s: 1.10.2
Component/s: scheduler
Labels:
None

Description

Airflow scheduler crashes if connectivity to Mysql is lost.

Below is the stack Trace
Traceback (most recent call last): File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 371, in helper pickle_dags) File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper result = func(*args, **kwargs) File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 1762, in process_file dag.sync_to_db() File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper result = func(*args, **kwargs) File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/models.py", line 3816, in sync_to_db session.commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 943, in commit self.transaction.commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 471, in commit t[1].commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1643, in commit self._do_commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1674, in _do_commit self.connection._commit_impl() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 726, in _commit_impl self._handle_dbapi_exception(e, None, None, None, None) File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception exc_info File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 724, in _commit_impl self.engine.dialect.do_commit(self.connection) File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/mysql/base.py", line 1784, in do_commit dbapi_connection.commit() OperationalError: (_mysql_exceptions.OperationalError) (2013, 'Lost connection to MySQL server during query') (Background on this error at: http://sqlalche.me/e/e3q8)
Process DagFileProcessor141318-Process: