Details
-
Bug
-
Status: Open
-
Critical
-
Resolution: Unresolved
-
4.5.2
-
None
-
Security Level: Public (Anyone can view this level - this is the default.)
-
None
-
Redundant Management Server Cluster, CentOS7. Two Zones, one for XEN, one for Hyper-V
Two xen hosts, two Hyper-V hosts
Description
Webconsole is not working with Hyper-V
Only a black Screen is showing up in console proxy browser window.
I already tried to debug it with:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/View+Console+and+Console+Proxy+Troubleshooting
Everything seems to be fine.
I also used the rdp-config.bat on the hosts:
https://github.com/apache/cloudstack/blob/master/services/console-proxy-rdp/rdpconsole/rdp-config.bat
Firewall on the hyperv hosts is off.
Logs Management Servers:
2015-09-24 17:01:19,387 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-6af52d42) Begin cleanup expired async-jobs
2015-09-24 17:01:19,392 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-6af52d42) End cleanup expired async-jobs
2015-09-24 17:01:21,100 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-13:null) SeqA 14-4610: Processing Seq 14-4610: { Cmd , MgmtId: -1, via: 14, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":75,"_loadInfo":"{\n \"connections\": [\n
\n ]\n}","wait":0}}] }
2015-09-24 17:01:21,103 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-13:null) SeqA 14-4610: Sending Seq 14-4610: { Ans: , MgmtId: 146454695956, via: 14, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2015-09-24 17:01:21,158 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-181:ctx-beeaa240) Seq 3-1841972247594567063: Executing request
2015-09-24 17:01:21,223 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-181:ctx-beeaa240) Vm cpu utilization 0.11527777777777777
2015-09-24 17:01:21,223 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-181:ctx-beeaa240) Seq 3-1841972247594567063: Response Received:
2015-09-24 17:01:21,223 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (DirectAgent-181:ctx-beeaa240) Seq 3-1841972247594567063: MgmtId 146454790969: Resp: Routing to peer
2015-09-24 17:01:21,243 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-372:ctx-ee7a486a) Seq 4-7128353785197397239: Executing request
2015-09-24 17:01:21,285 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-372:ctx-ee7a486a) Vm cpu utilization 0.19722222222222224
2015-09-24 17:01:21,285 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-372:ctx-ee7a486a) Vm cpu utilization 0.11638888888888888
2015-09-24 17:01:21,285 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-372:ctx-ee7a486a) Vm cpu utilization 0.20611111111111113
2015-09-24 17:01:21,285 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-372:ctx-ee7a486a) Seq 4-7128353785197397239: Response Received:
2015-09-24 17:01:21,286 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (DirectAgent-372:ctx-ee7a486a) Seq 4-7128353785197397239: MgmtId 146454790969: Resp: Routing to peer
2015-09-24 17:01:21,302 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-370:ctx-f4af3477) Seq 11-5374201730336556383: Executing request
2015-09-24 17:01:21,303 DEBUG [c.c.h.h.r.HypervDirectConnectResource] (DirectAgent-370:ctx-f4af3477) POST request to https://192.168.4.63:8250/api/HypervResource/com.cloud.agent.api.GetVmStatsCommand with contents {"vmNames":["i-2-81-VM"],"hostGuid":"6fa76640-72db-38ba-a034-5d70a0d43fc6-HypervResource","hostName":"192.168.4.63","contextMap":{},"wait":0}
2015-09-24 17:01:21,306 DEBUG [c.c.h.h.r.HypervDirectConnectResource] (DirectAgent-370:ctx-f4af3477) Sending cmd to https://192.168.4.63:8250/api/HypervResource/com.cloud.agent.api.GetVmStatsCommand cmd data:{"vmNames":["i-2-81-VM"],"hostGuid":"6fa76640-72db-38ba-a034-5d70a0d43fc6-HypervResource","hostName":"192.168.4.63","contextMap":{},"wait":0}
2015-09-24 17:01:21,580 DEBUG [c.c.h.h.r.HypervDirectConnectResource] (DirectAgent-370:ctx-f4af3477) POST response is [{"com.cloud.agent.api.GetVmStatsAnswer":{"vmStatsMap":{"i-2-81-VM":{"cpuUtilization":0.0,"networkReadKBs":1.0,"networkWriteKBs":1.0,"numCPUs":1,"entityType":"vm"}},"result":true,"contextMap":{}}}]
2015-09-24 17:01:21,581 DEBUG [c.c.h.h.r.HypervDirectConnectResource] (DirectAgent-370:ctx-f4af3477) executeRequest received response [{"com.cloud.agent.api.GetVmStatsAnswer":{"vmStatsMap":{"i-2-81-VM":{"cpuUtilization":0.0,"networkReadKBs":1.0,"networkWriteKBs":1.0,"diskReadIOs":0.0,"diskWriteIOs":0.0,"diskReadKBs":0.0,"diskWriteKBs":0.0,"numCPUs":1,"entityType":"vm"}},"result":true,"contextMap":{},"wait":0}}]
2015-09-24 17:01:21,581 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-370:ctx-f4af3477) Seq 11-5374201730336556383: Response Received:
2015-09-24 17:01:21,581 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (DirectAgent-370:ctx-f4af3477) Seq 11-5374201730336556383: MgmtId 146454790969: Resp: Routing to peer
2015-09-24 17:01:22,119 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-5:null) SeqA 5-15081: Processing Seq 5-15081: { Cmd , MgmtId: -1, via: 5, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"
","wait":0}}] }
2015-09-24 17:01:22,121 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-5:null) SeqA 5-15081: Sending Seq 5-15081: { Ans: , MgmtId: 146454695956, via: 5, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
146454790969, via: 11(192.168.4.63), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.GetVncPortCommand":{"id":81,"name":"i-2-81-VM","wait":0}}] } to 146454695956
2015-09-24 17:01:10,169 DEBUG [c.c.a.t.Request] (AgentManager-Handler-9:null) Seq 11-5374201730336556382: Processing: { Ans: , MgmtId: 146454790969, via: 11, Ver: v1, Flags: 10, [{"com.cloud.agent.api.GetVncPortAnswer":{"address":"instanceId=9D626339-6969-407D-98F7-51308B3F5F47","port":-9,"result":true,"wait":0}}] }
2015-09-24 17:01:10,170 DEBUG [c.c.a.t.Request] (catalina-exec-5:null) Seq 11-5374201730336556382: Received: { Ans: , MgmtId: 146454790969, via: 11, Ver: v1, Flags: 10,
}
2015-09-24 17:01:10,170 DEBUG [c.c.s.ConsoleProxyServlet] (catalina-exec-5:null) Port info instanceId=9D626339-6969-407D-98F7-51308B3F5F47
2015-09-24 17:01:10,170 INFO [c.c.s.ConsoleProxyServlet] (catalina-exec-5:null) Parse host info returned from executing GetVNCPortCommand. host info: instanceId=9D626339-6969-407D-98F7-51308B3F5F47
2015-09-24 17:01:10,178 DEBUG [c.c.s.ConsoleProxyServlet] (catalina-exec-5:null) Compose console url: https://URL/ajax?token=FJghDTzoZqc4s790v0M2FtEcKH02fhsuLCPxS0-uzFN9bBdrDI34LvcIgExHHZxPbvRBDNhN8B6tGXNgykE5oiw7_cFZ2MOeKtrSzBvm6a2CGRRc3ZqPspgRuTjgOaqDnYZXVUgJ5KWKQmywPqTCirB7sj97pYFMCyIc0kDVEN8xaEYctBcn9Mn9aB5nLFul-ox3dZlSFIjU4Q87KZnOwt2akYGeuMxzaoIYCPrSEixHUFWRJwqtF2gW34pYb3j6ZEdqYYcKEN-8804HV-_oB37WX93e6hjTsqlu9RW6C2O7AUODuoELOlSTWfz6xyKpl1WJlcfI6DI4wYpuTr7o4gq_BIVtrO9ZRzk5aQoPlddod4HFD415ShhMHFnJhmC7vbsc6aMTy1hb24Hwzldp5cYpfQ7u9P26-EOIIk28o88
2015-09-24 17:01:10,178 DEBUG [c.c.s.ConsoleProxyServlet] (catalina-exec-5:null) the console url is :: <html><title>hyperv-vm1</title><frameset><frame src="https://URL/ajax?token=FJghDTzoZqc4s790v0M2FtEcKH02fhsuLCPxS0-uzFN9bBdrDI34LvcIgExHHZxPbvRBDNhN8B6tGXNgykE5oiw7_cFZ2MOeKtrSzBvm6a2CGRRc3ZqPspgRuTjgOaqDnYZXVUgJ5KWKQmywPqTCirB7sj97pYFMCyIc0kDVEN8xaEYctBcn9Mn9aB5nLFul-ox3dZlSFIjU4Q87KZnOwt2akYGeuMxzaoIYCPrSEixHUFWRJwqtF2gW34pYb3j6ZEdqYYcKEN-8804HV-_oB37WX93e6hjTsqlu9RW6C2O7AUODuoELOlSTWfz6xyKpl1WJlcfI6DI4wYpuTr7o4gq_BIVtrO9ZRzk5aQoPlddod4HFD415ShhMHFnJhmC7vbsc6aMTy1hb24Hwzldp5cYpfQ7u9P26-EOIIk28o88"></frame></frameset></html>
Logs console proxy:
2015-09-24 15:06:16,120 DEBUG [cloud.consoleproxy.ConsoleProxyGCThread] (Console Proxy GC Thread:null) connMap=
2015-09-24 15:06:19,148 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1215:null) decode token. host: 14CD2217-1F88-44C3-B026-7E5B155C8663
2015-09-24 15:06:19,148 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1215:null) decode token. port: 2179
2015-09-24 15:06:19,148 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1215:null) decode token. tag: 9dfbb1bc-a9d1-4356-9d95-389c93b1451d
2015-09-24 15:06:19,197 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null) Received response: Seq 14-4657: { Ans: , MgmtId: 146454695956, via: 14, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.ConsoleAccessAuthenticationAnswer":{"_success":true,"_isReauthenticating":false,"_port":0,"result":true,"wait":0}}] }
2015-09-24 15:06:19,203 INFO [cloud.consoleproxy.ConsoleProxyRdpClient] (Thread-1215:null) connecting to instance 14CD2217-1F88-44C3-B026-7E5B155C8663 on host 192.168.4.63
2015-09-24 15:06:19,205 INFO [cloud.consoleproxy.ConsoleProxy] (Thread-1215:null) Added viewer object com.cloud.consoleproxy.ConsoleProxyRdpClient@37a417e7
2015-09-24 15:06:19,206 DEBUG [resource.consoleproxy.ConsoleProxyResource] (Thread-1215:null) Report proxy load info, proxy : 75, load: {
"connections": [
,
{ "id": 7, "clientInfo": "", "host": "14CD2217-1F88-44C3-B026-7E5B155C8663", "port": 2179, "tag": "9dfbb1bc-a9d1-4356-9d95-389c93b1451d", "createTime": 1443107179198, "lastUsedTime": 1443107179198 } ]
}
2015-09-24 15:06:19,206 DEBUG [cloud.consoleproxy.ConsoleProxy] (Thread-1215:null) Report load change : {
"connections": [
,
{ "id": 7, "clientInfo": "", "host": "14CD2217-1F88-44C3-B026-7E5B155C8663", "port": 2179, "tag": "9dfbb1bc-a9d1-4356-9d95-389c93b1451d", "createTime": 1443107179198, "lastUsedTime": 1443107179198 } ]
}
2015-09-24 15:06:19,206 DEBUG [cloud.consoleproxy.ConsoleProxyAjaxHandler] (Thread-1215:null) Ajax request indicates a fresh client start
2015-09-24 15:06:19,210 DEBUG [resource.consoleproxy.ConsoleProxyResource] (Thread-1216:null) Ensure route for 192.168.4.63 via 192.168.4.1
2015-09-24 15:06:19,212 DEBUG [resource.consoleproxy.ConsoleProxyResource] (Thread-1216:null) addRouteToInternalIp: localgw=192.168.4.1, eth1ip=192.168.5.122, eth1mask=255.255.252.0,destIp=192.168.4.63
2015-09-24 15:06:19,212 DEBUG [resource.consoleproxy.ConsoleProxyResource] (Thread-1216:null) addRouteToInternalIp: dest ip 192.168.4.63 is in the same subnet as eth1 ip 192.168.5.122
2015-09-24 15:06:19,213 INFO [cloud.consoleproxy.ConsoleProxyRdpClient] (Thread-1216:null) Connecting socket to remote server and run main loop(s)
2015-09-24 15:06:19,290 DEBUG [cloud.agent.Agent] (Agent-Handler-2:null) Received response: Seq 14-4658: { Ans: , MgmtId: 146454695956, via: 14, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2015-09-24 15:06:19,377 DEBUG [cloud.consoleproxy.ConsoleProxyAjaxImageHandler] (Thread-1218:null) AjaxImageHandler /ajaximg?token=FJghDTzoZqc4s790v0M2FmTgGJfRH4fA8s_EBqrHa3IwcQYq_8mfUmLjc1_fYy7rwLR9HBK2WGY6fkqeTT76dVkfx5zlN9QvGx8v-CPWO2nAG9HrWEm8xKSu8B1rOiHAKzUAqQYU24jF7-j2LvFqqhfY1Kovi1RtlV-ctp6fius_Ryj08gzauCJUh43Y3leK_wuL3qhES7S14es9apd0NKZSW7_QE6rFlmE5cU-6FByOigLeb3TMMOeBgoWCXAZZcdojDCvGWtk2nTtWf1IkIzc58wcxOun607cPGPEqx9HwVQnJHq_wCkVKsHH7zWXDlBcIWKDI25_PXZ9giJVoW65S5LW2csiEYVmOQML_3BZU5fijYE0996EIkxNRCIC_eDvXFGv5ppy-8mDwRtOeCXG-cFleVzKk6PmLFmUUvEw&key=1&ts=1443107179274
2015-09-24 15:06:19,379 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1218:null) decode token. host: 14CD2217-1F88-44C3-B026-7E5B155C8663
2015-09-24 15:06:19,379 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1218:null) decode token. port: 2179
2015-09-24 15:06:19,379 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1218:null) decode token. tag: 9dfbb1bc-a9d1-4356-9d95-389c93b1451d
2015-09-24 15:06:19,382 DEBUG [cloud.consoleproxy.ConsoleProxyAjaxImageHandler] (Thread-1218:null) /ajaximg?token=FJghDTzoZqc4s790v0M2FmTgGJfRH4fA8s_EBqrHa3IwcQYq_8mfUmLjc1_fYy7rwLR9HBK2WGY6fkqeTT76dVkfx5zlN9QvGx8v-CPWO2nAG9HrWEm8xKSu8B1rOiHAKzUAqQYU24jF7-j2LvFqqhfY1Kovi1RtlV-ctp6fius_Ryj08gzauCJUh43Y3leK_wuL3qhES7S14es9apd0NKZSW7_QE6rFlmE5cU-6FByOigLeb3TMMOeBgoWCXAZZcdojDCvGWtk2nTtWf1IkIzc58wcxOun607cPGPEqx9HwVQnJHq_wCkVKsHH7zWXDlBcIWKDI25_PXZ9giJVoW65S5LW2csiEYVmOQML_3BZU5fijYE0996EIkxNRCIC_eDvXFGv5ppy-8mDwRtOeCXG-cFleVzKk6PmLFmUUvEw&key=1&ts=1443107179274Process time 4 ms
2015-09-24 15:06:20,165 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1219:null) decode token. host: 14CD2217-1F88-44C3-B026-7E5B155C8663
2015-09-24 15:06:20,165 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1219:null) decode token. port: 2179
2015-09-24 15:06:20,165 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1219:null) decode token. tag: 9dfbb1bc-a9d1-4356-9d95-389c93b1451d
2015-09-24 15:06:20,384 DEBUG [cloud.consoleproxy.ConsoleProxyAjaxImageHandler] (Thread-1220:null) AjaxImageHandler /ajaximg?token=FJghDTzoZqc4s790v0M2FmTgGJfRH4fA8s_EBqrHa3IwcQYq_8mfUmLjc1_fYy7rwLR9HBK2WGY6fkqeTT76dVkfx5zlN9QvGx8v-CPWO2nAG9HrWEm8xKSu8B1rOiHAKzUAqQYU24jF7-j2LvFqqhfY1Kovi1RtlV-ctp6fius_Ryj08gzauCJUh43Y3leK_wuL3qhES7S14es9apd0NKZSW7_QE6rFlmE5cU-6FByOigLeb3TMMOeBgoWCXAZZcdojDCvGWtk2nTtWf1IkIzc58wcxOun607cPGPEqx9HwVQnJHq_wCkVKsHH7zWXDlBcIWKDI25_PXZ9giJVoW65S5LW2csiEYVmOQML_3BZU5fijYE0996EIkxNRCIC_eDvXFGv5ppy-8mDwRtOeCXG-cFleVzKk6PmLFmUUvEw&key=2&ts=1443107180168
2015-09-24 15:06:20,385 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1220:null) decode token. host: 14CD2217-1F88-44C3-B026-7E5B155C8663
2015-09-24 15:06:20,385 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1220:null) decode token. port: 2179
2015-09-24 15:06:20,385 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1220:null) decode token. tag: 9dfbb1bc-a9d1-4356-9d95-389c93b1451d
2015-09-24 15:06:20,387 DEBUG [cloud.consoleproxy.ConsoleProxyAjaxImageHandler] (Thread-1220:null) /ajaximg?token=FJghDTzoZqc4s790v0M2FmTgGJfRH4fA8s_EBqrHa3IwcQYq_8mfUmLjc1_fYy7rwLR9HBK2WGY6fkqeTT76dVkfx5zlN9QvGx8v-CPWO2nAG9HrWEm8xKSu8B1rOiHAKzUAqQYU24jF7-j2LvFqqhfY1Kovi1RtlV-ctp6fius_Ryj08gzauCJUh43Y3leK_wuL3qhES7S14es9apd0NKZSW7_QE6rFlmE5cU-6FByOigLeb3TMMOeBgoWCXAZZcdojDCvGWtk2nTtWf1IkIzc58wcxOun607cPGPEqx9HwVQnJHq_wCkVKsHH7zWXDlBcIWKDI25_PXZ9giJVoW65S5LW2csiEYVmOQML_3BZU5fijYE0996EIkxNRCIC_eDvXFGv5ppy-8mDwRtOeCXG-cFleVzKk6PmLFmUUvEw&key=2&ts=1443107180168Process time 3 ms
2015-09-24 15:06:20,599 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1221:null) decode token. host: 14CD2217-1F88-44C3-B026-7E5B155C8663
2015-09-24 15:06:20,600 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1221:null) decode token. port: 2179
2015-09-24 15:06:20,600 DEBUG [cloud.consoleproxy.ConsoleProxyHttpHandlerHelper] (Thread-1221:null) decode token. tag: 9dfbb1bc-a9d1-4356-9d95-389c93b1451d
2015-09-24 15:06:21,121 DEBUG [cloud.consoleproxy.ConsoleProxyGCThread] (Console Proxy GC Thread:null) connMap=
2015-09-24 15:06:21,122 DEBUG [resource.consoleproxy.ConsoleProxyResource] (Console Proxy GC Thread:null) Report proxy load info, proxy : 75, load: {
"connections": [
,
{ "id": 7, "clientInfo": "", "host": "14CD2217-1F88-44C3-B026-7E5B155C8663", "port": 2179, "tag": "9dfbb1bc-a9d1-4356-9d95-389c93b1451d", "createTime": 1443107179198, "lastUsedTime": 1443107180600 } ]
}
2015-09-24 15:06:21,123 DEBUG [cloud.consoleproxy.ConsoleProxyGCThread] (Console Proxy GC Thread:null) Report load change : {
"connections": [
,
{ "id": 7, "clientInfo": "", "host": "14CD2217-1F88-44C3-B026-7E5B155C8663", "port": 2179, "tag": "9dfbb1bc-a9d1-4356-9d95-389c93b1451d", "createTime": 1443107179198, "lastUsedTime": 1443107180600 } ]
}
2015-09-24 15:06:21,169 DEBUG [cloud.agent.Agent] (Agent-Handler-3:null) Received response: Seq 14-4659: { Ans: , MgmtId: 146454695956, via: 14, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
WebConsole on XEN Zone is working fine.
What could be the problem, how can i debug it further?