While running in secure mode, supervisor sets the worker user (in workers local state) as the user that launched the topology.
However the worker OS process does not actually run as the user "foo" (instead runs as storm user) unless supervisor.run.worker.as.user is also set.
If the supervisor's assignment changes, the supervisor in some cases checks if all processes are dead by matching the "pid+user". Here if the worker is running as a different user (say storm) the supervisor wrongly assumes that the worker process is dead.
Later when supervisor tries to launch a worker at that same port, it throws a bind exception
o.a.s.m.n.Server main [INFO] Create Netty Server Netty-server-localhost-6700, buffer_size: 5242880, maxWorkers: 1
o.a.s.d.worker main [ERROR] Error on initialization of server mk-worker
org.apache.storm.shade.org.jboss.netty.channel.ChannelException: Failed to bind to: 0.0.0.0/0.0.0.0:6700
at org.apache.storm.shade.org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272) ~[storm-core-126.96.36.199.1.0.0-501.jar:188.8.131.52.1.0.0-501]