[CLOUDSTACK-6496] addHost fails for XenServer with vSwitch networking

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Not A Problem
• Affects Version/s: 4.4.0, Future
• Fix Version/s: 4.5.0
• Component/s: XenServer
• Security Level: Public (Anyone can view this level - this is the default.)
• Environment: MS: ACS Master (http://jenkins.buildacloud.org/job/package-rhel63-master/2647); XenServer 6.2
• Assignee: Anthony Xu
• Reporter: Doug Clark
• Created: 24/Apr/14 14:19
• Updated: 15/Oct/14 22:31
• Resolved: 15/Oct/14 22:31

Description

Attempt to add a XenServer host (with the default vSwitch networking) to a Basic Networking Zone fails. Adding a XenServer host configured to use bridge works ok.

From MS log (attached):

2014-04-24 13:41:07,361 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-1:ctx-3e360a0c) Failed to configure brige firewall
2014-04-24 13:41:07,361 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-1:ctx-3e360a0c) Check host 10.81.40.102 for CSP is installed or not and check network mode for bridge
2014-04-24 13:41:07,361 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-1:ctx-3e360a0c) Seq 1-7133701809754865665: Response Received:
2014-04-24 13:41:07,363 DEBUG [c.c.a.t.Request] (DirectAgent-1:ctx-3e360a0c) Seq 1-7133701809754865665: Processing:  { Ans: , MgmtId: 275410316893143, via: 1, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Set
upAnswer":{"_reconnect":true,"result":false,"details":"Failed to configure brige firewall","wait":0}}] }
2014-04-24 13:41:07,363 DEBUG [c.c.a.t.Request] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Seq 1-7133701809754865665: Received:  { Ans: , MgmtId: 275410316893143, via: 1, Ver: v1, Flags: 110,
{ SetupAnswer } }
2014-04-24 13:41:07,363 WARN  [c.c.h.x.d.XcpServerDiscoverer] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Unable to setup agent 1 due to Failed to configure brige firewall
2014-04-24 13:41:07,364 INFO  [c.c.u.e.CSExceptionErrorCode] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Could not find exception: com.cloud.exception.ConnectionException in error code list for
 exceptions
2014-04-24 13:41:07,364 WARN  [c.c.a.m.AgentManagerImpl] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Monitor XcpServerDiscoverer says there is an error in the connect process for 1 due to Reini
tialize agent after setup.
2014-04-24 13:41:07,364 INFO  [c.c.a.m.AgentManagerImpl] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Host 1 is disconnecting with event AgentDisconnected
2014-04-24 13:41:07,364 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-1:ctx-3e360a0c) Seq 1-7133701809754865665: No more commands found
2014-04-24 13:41:07,366 DEBUG [c.c.a.m.AgentManagerImpl] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) The next status of agent 1is Alert, current status is Connecting
2014-04-24 13:41:07,366 DEBUG [c.c.a.m.AgentManagerImpl] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Deregistering link for 1 with state Alert
2014-04-24 13:41:07,366 DEBUG [c.c.a.m.AgentManagerImpl] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Remove Agent : 1

...snip...

2014-04-24 13:41:07,460 DEBUG [c.c.a.m.AgentManagerImpl] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Sending Disconnect to listener: com.cloud.network.router.VirtualNetworkApplianceManagerImpl
2014-04-24 13:41:07,460 DEBUG [c.c.h.Status] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host id = 1, name = xrtuk-09-03]
2014-04-24 13:41:07,766 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Notifying other nodes of to disconnect
2014-04-24 13:41:07,770 WARN  [c.c.r.ResourceManagerImpl] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Unable to connect due to
com.cloud.exception.ConnectionException: Reinitialize agent after setup.
        at com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer.processConnect(XcpServerDiscoverer.java:657)
        at com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:514)
        at com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(AgentManagerImpl.java:1428)
        at com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1767)
        at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:773)
        at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:586)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at com.sun.proxy.$Proxy146.discoverHosts(Unknown Source)
        at org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:142)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:134)
        at com.cloud.api.ApiServer.queueCommand(ApiServer.java:593)
        at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
        at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:317)
        at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
        at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
2014-04-24 13:41:07,772 DEBUG [c.c.h.Status] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host id = 1, name = xrtuk-09-03]
2014-04-24 13:41:07,772 DEBUG [c.c.h.Status] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) Cannot transit agent status with event AgentDisconnected for host 1, name=xrtuk-09-03, mangement server id is 275410316893143
2014-04-24 13:41:07,772 ERROR [c.c.a.ApiServer] (catalina-exec-2:ctx-407da4e1 ctx-e02434c0 ctx-a13beb18) unhandled exception executing api command: [Ljava.lang.String;@6b243b82
com.cloud.utils.exception.CloudRuntimeException: Cannot transit agent status with event AgentDisconnected for host 1, mangement server id is 275410316893143,Unable to transition to a new state from Creating via AgentDisconnected

Issue Links

• is cloned by: CLOUDSTACK-7734 CLONE - addHost fails for XenServer with vSwitch networking