[METRON-353] Monit unable to start mysqld after successful single node vagrant deployment

• Type: Bug
• Status: Done
• Priority: Major
• Resolution: Done
• Affects Version/s: 0.2.1BETA
• Labels: metronqe
• Environment: Quick dev setup installation using vagrant on Macbook Pro.
• Assignee: Unassigned
• Reporter: Anand Subramanian
• Created: 26/Jul/16 06:20
• Updated: 31/Oct/17 11:08
• Resolved: 31/Oct/17 11:08

Description

This is a single node quick-dev deployment from HEAD using vagrant on a Macbook Pro. The platform-info output is attached.

I noticed that monit was not able to start despite trying multiple times and I had to start it manually post-deployment.

Note that post-installation, I am able to manually start the mysqld service and it comes up fine.

Here are the relevant lines from the /var/log/messages on the vagrant node.

Jul 25 10:30:11 node1 ansible-command: Invoked with warn=True executable=None chdir=None _raw_params=monit start mysql removes=None creates=None _uses_shell=True
<snip>
Jul 25 10:30:21 node1 ansible-command: Invoked with warn=True executable=None chdir=None _raw_params=monit start mysql removes=None creates=None _uses_shell=True
Jul 25 10:30:32 node1 ansible-command: Invoked with warn=True executable=None chdir=None _raw_params=monit start mysql removes=None creates=None _uses_shell=True
Jul 25 10:30:42 node1 ansible-command: Invoked with warn=True executable=None chdir=None _raw_params=monit start mysql removes=None creates=None _uses_shell=True
Jul 25 10:30:52 node1 ansible-command: Invoked with warn=True executable=None chdir=None _raw_params=monit start mysql removes=None creates=None _uses_shell=True
Jul 25 10:31:02 node1 ansible-command: Invoked with warn=True executable=None chdir=None _raw_params=monit start mysql removes=None creates=None _uses_shell=True
<snip>

From the /var/log/mysqld.log, I could see that there was a shutdown initiated after monit tried restarting bunch of times:

2016-07-25T10:30:55.152359Z 0 [Note] Giving 0 client threads a chance to die gracefully
2016-07-25T10:30:55.152397Z 0 [Note] Shutting down slave threads
2016-07-25T10:30:55.152405Z 0 [Note] Forcefully disconnecting 0 remaining clients
2016-07-25T10:30:55.152471Z 0 [Note] Event Scheduler: Purging the queue. 0 events
2016-07-25T10:30:55.153536Z 0 [Note] Binlog end
2016-07-25T10:30:55.154549Z 0 [Note] Shutting down plugin 'validate_password'
2016-07-25T10:30:55.154573Z 0 [Note] Shutting down plugin 'ngram'
2016-07-25T10:30:55.154577Z 0 [Note] Shutting down plugin 'ARCHIVE'
2016-07-25T10:30:55.154581Z 0 [Note] Shutting down plugin 'partition'
2016-07-25T10:30:55.154584Z 0 [Note] Shutting down plugin 'BLACKHOLE'
2016-07-25T10:30:55.154588Z 0 [Note] Shutting down plugin 'CSV'
2016-07-25T10:30:55.154601Z 0 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2016-07-25T10:30:55.154621Z 0 [Note] Shutting down plugin 'MyISAM'
2016-07-25T10:30:55.154631Z 0 [Note] Shutting down plugin 'INNODB_SYS_VIRTUAL'
2016-07-25T10:30:55.154635Z 0 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2016-07-25T10:30:55.154638Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES'
2016-07-25T10:30:55.154640Z 0 [Note] Shutting down plugin ‘INNODB_SYS_FOREIGN_COLS'
<snip>