I have setup the storm cluster in AWS environment with 6 node clusters. 1 Master and 5 slave and the zookeeper are installed in 3 node clusters.
When I shutdown zookeeper and storm nodes every day night and start in the morning, When the machine boots up I have started all the nimbus and supervisor and zookeeper service via supervisord. Once machine up all the service are running but in the nimbus logs I am getting "java.lang.RuntimeException: No nimbus leader participant host found, have you started your nimbus hosts?" error.
I have restarted zookeeper and nimbus also not working still getting this error. Once I remove all the data from zookeeper data directly then restart zookeeper and storm service then working fine.
Why I have to do this everyday when the machine starts up? how can I fix this issue. When machine boot up all the service should work fine.