[AIRFLOW-4050] MySQL tests fail against Mysql8.0

• Type: Test
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: database
• Labels: mysql
• Assignee: Unassigned
• Reporter: Ash Berlin-Taylor
• Created: 08/Mar/19 12:06
• Updated: 29/May/19 03:34

Description

I tried running tests against mysql docker image which happened to be 8.0 and got the following errors:

======================================================================
2) ERROR: test_mysql_hook_test_bulk_load (tests.operators.test_operators.MySqlTest)
----------------------------------------------------------------------
   Traceback (most recent call last):
    tests/operators/test_operators.py line 106 in test_mysql_hook_test_bulk_load
      h.bulk_load("test_airflow", t.name)
    airflow/hooks/mysql_hook.py line 115 in bulk_load
      """.format(**locals()))
    /usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py line 255 in execute
      self.errorhandler(self, exc, value)
    /usr/local/lib/python3.6/site-packages/MySQLdb/connections.py line 50 in defaulterrorhandler
      raise errorvalue
    /usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py line 252 in execute
      res = self._query(query)
    /usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py line 378 in _query
      db.query(q)
    /usr/local/lib/python3.6/site-packages/MySQLdb/connections.py line 280 in query
      _mysql.connection.query(self, query)
   OperationalError: (1148, 'The used command is not allowed with this MySQL version')
   -------------------- >> begin captured logging << --------------------
   airflow.utils.log.logging_mixin.LoggingMixin: INFO: Using connection to: id: airflow_db. Host: 172.17.0.3, Port: None, Schema: airflow, Login: root, Password: XXXXXXXX, extra: {}
   airflow.utils.log.logging_mixin.LoggingMixin: INFO: Using connection to: id: airflow_db. Host: 172.17.0.3, Port: None, Schema: airflow, Login: root, Password: XXXXXXXX, extra: {}
   airflow.utils.log.logging_mixin.LoggingMixin: INFO: Using connection to: id: mysql_default. Host: 172.17.0.3, Port: None, Schema: airflow, Login: root, Password: XXXXXXXX, extra: {}
   --------------------- >> end captured logging << ---------------------
======================================================================
3) ERROR: test_mysql_to_mysql (tests.operators.test_operators.MySqlTest)
----------------------------------------------------------------------
   Traceback (most recent call last):
    tests/operators/test_operators.py line 162 in test_mysql_to_mysql
      t.run(start_date=DEFAULT_DATE, end_date=DEFAULT_DATE, ignore_ti_state=True)
    airflow/models.py line 3067 in run
      ignore_ti_state=ignore_ti_state)
    airflow/utils/db.py line 73 in wrapper
      return func(*args, **kwargs)
    airflow/models.py line 1765 in run
      session=session)
    airflow/utils/db.py line 69 in wrapper
      return func(*args, **kwargs)
    airflow/models.py line 1672 in _run_raw_task
      result = task_copy.execute(context=context)
    airflow/operators/generic_transfer.py line 78 in execute
      destination_hook.run(self.preoperator)
    airflow/hooks/dbapi_hook.py line 170 in run
      cur.execute(s)
    /usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py line 255 in execute
      self.errorhandler(self, exc, value)
    /usr/local/lib/python3.6/site-packages/MySQLdb/connections.py line 50 in defaulterrorhandler
      raise errorvalue
    /usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py line 252 in execute
      res = self._query(query)
    /usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py line 378 in _query
      db.query(q)
    /usr/local/lib/python3.6/site-packages/MySQLdb/connections.py line 280 in query
      _mysql.connection.query(self, query)
   OperationalError: (1347, "'information_schema.TABLES' is not BASE TABLE")
   -------------------- >> begin captured logging << --------------------
   airflow.utils.log.logging_mixin.LoggingMixin: INFO: Using connection to: id: airflow_db. Host: 172.17.0.3, Port: None, Schema: airflow, Login: root, Password: XXXXXXXX, extra: {}
   airflow.utils.log.logging_mixin.LoggingMixin: INFO: Using connection to: id: airflow_db. Host: 172.17.0.3, Port: None, Schema: airflow, Login: root, Password: XXXXXXXX, extra: {}
   airflow.utils.log.logging_mixin.LoggingMixin: INFO: Using connection to: id: airflow_db. Host: 172.17.0.3, Port: None, Schema: airflow, Login: root, Password: XXXXXXXX, extra: {}
   airflow.utils.log.logging_mixin.LoggingMixin: INFO: Using connection to: id: airflow_db. Host: 172.17.0.3, Port: None, Schema: airflow, Login: root, Password: XXXXXXXX, extra: {}
   airflow.hooks.mysql_hook.MySqlHook: INFO: DROP TABLE IF EXISTS test_mysql_to_mysql
   airflow.hooks.mysql_hook.MySqlHook: INFO: CREATE TABLE IF NOT EXISTS test_mysql_to_mysql LIKE INFORMATION_SCHEMA.TABLES
   airflow.utils.log.logging_mixin.LoggingMixin: INFO: Using connection to: id: mysql_default. Host: 172.17.0.3, Port: None, Schema: airflow, Login: root, Password: XXXXXXXX, extra: {}
   --------------------- >> end captured logging << ---------------------

Issue Links

• relates to: AIRFLOW-4051 MySQL hook bulk_load function doesn't work by default with MySql 8