Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
4.1.1, 4.2.0, 4.4.0
-
Security Level: Public (Anyone can view this level - this is the default.)
-
None
-
CentOS 6.4, Ubuntu 14.04
Description
Installing CloudStack 4.1.1 today, the management server failed to start with the following errors:
2013-09-30 14:43:43,436 INFO [utils.component.ComponentContext] (Timer-2:null) Running SystemIntegrityChecker managementServerNode
2013-09-30 14:43:43,438 ERROR [utils.component.ComponentContext] (Timer-2:null) System integrity check failed. Refuse to startup
Due to CLOUDSTACK-4170, there wasn't any useful information provided, so I attempted to reinstall with CloudStack 4.2.0. This produced the following error messages:
2013-09-30 14:43:43,436 INFO [utils.component.ComponentContext] (Timer-2:null) Running SystemIntegrityChecker managementServerNode
2013-09-30 14:43:43,438 ERROR [utils.component.ComponentContext] (Timer-2:null) System integrity check failed. Refuse to startup
com.cloud.utils.exception.CloudRuntimeException: Unable to get the management server node id
at com.cloud.cluster.ManagementServerNode.check(ManagementServerNode.java:46)
at com.cloud.utils.component.ComponentContext.initComponentsLifeCycle(ComponentContext.java:90)
at com.cloud.servlet.CloudStartupServlet$1.run(CloudStartupServlet.java:54)
at java.util.TimerThread.mainLoop(Timer.java:534)
at java.util.TimerThread.run(Timer.java:484)
I found a mailing list post which mentioned that this might be due to missing MAC address, and recommended testing by running the class directly. This produced the following output for me:
$ java -classpath /usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-utils-4.2.0.jar com.cloud.utils.net.MacAddress
addr in integer is 0
addr in bytes is 0 0 0 0 0 0
addr in char is 00:00:00:00:00:00
This was odd to me, as the output of ifconfig didn't have any MAC addresses that were 00:00:00:00:00:00. Looking into the code for that module, I found that what it appears to be doing is parsing ifconfig -a and returning the first string that looks like a MAC address. Reviewing the ifconfig -a output on the server, there was a downed bond interface with a MAC address of 00:00:00:00:00:00. When this interface is up, it automatically gets a MAC address from one of its slave physical interfaces.
After assigning a dummy IP address to that interface and upping it, it received a MAC address and I was able to start the management server normally.
I expect that there may be a better way to determine what the node id for the management server should be, or logic could be implemented to look for another MAC address if the first once it receives is invalid. Alternatively, if this behavior is expected, logging/debugging resources should be provided to help the user correct the problem.