Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
4.5.2
-
None
-
None
-
Security Level: Public (Anyone can view this level - this is the default.)
-
None
Description
When using the migrateVirtualMachineWithVolume, in a clustered management server environment, migrations intermittently fail. This appears to be a similar issue to this jira ticket:
https://issues.apache.org/jira/browse/CLOUDSTACK-8412
For reference, the error is:
ERROR [c.c.a.t.Request] (AgentManager-Handler-11:null) Caught problem with [{"com.cloud.agent.api.MigrateWithStorageCommand":{"vm":{"id":26631,"name":"i-2-26631-VM","bootloader":"PyGrub","type":"Us
er","cpus":1,"minSpeed":1200,"maxSpeed":1200,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"CentOS 5.6 (64-bit)","platformEmulator":"CentOS 5 (64-bit)","bootArgs":"","enableHA":false,"
limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"","params":{"memoryOvercommitRatio":"1","platform":"viridian:true;acpi:1;apic:true;pae:true;nx:t
rue","hypervisortoolsversion":"xenserver56","cpuOvercommitRatio":"4"},"uuid":"b37e76e8-e19a-4100-bf1c-1ee8eba674f2","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"8f51
e73f-9bfc-4a37-bf86-664067a2cf40","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"b114d58e-a37a-6433-c0ff-07f3613a3c68","id":219,"poolType":"LVM","ho
st":"192.168.97.2","path":"lvm","port":0,"url":"LVM://192.168.97.2/lvm/?ROLE\u003dPrimary\u0026STOREUUID\u003db114d58e-a37a-6433-c0ff-07f3613a3c68"}},"name":"ROOT-26631","size":21474836480,"path":"48
fe494b-ee02-4f08-b7aa-975baecd7b3e","volumeId":119075,"vmName":"i-2-26631-VM","accountId":2,"format":"VHD","provisioningType":"THIN","id":119075,"deviceId":0,"cacheMode":"NONE","hypervisorType":"Xe
nServer"}},"diskSeq":0,"path":"48fe494b-ee02-4f08-b7aa-975baecd7b3e","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"192.168.97.2","volumeSize":"21474836480"}},{"data":{
"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":2000,"defaultNic":true,"pxe
Disable":false,"nicUuid":"aaef8c14-784c-403f-91fc-b6a3167f2595","uuid":"508d0384-5bf0-4434-8155-0b5bb1b33b43","ip":"192.168.8.112","netmask":"255.255.254.0","gateway":"192.168.9.254","mac":"06:68:80:
00:00:7f","dns1":"192.168.0.224","dns2":"192.168.0.224","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true,"name":"CS-Guest-Public"}],"vcpuMaxLimit":
16},"volumeToFiler":{"Vol[119075|ROOT|48fe494b-ee02-4f08-b7aa-975baecd7b3e|21474836480]":{"id":223,"uuid":"bc01ca2b-e6d8-a856-27b5-6f6b55a06b1d","host":"192.168.97.1","path":"lvm","port":0,"type":"L
VM"}},"contextMap":{"job":"job-347068/job-347069"},"wait":0}}]
com.google.gson.JsonParseException: Expecting object found: "Vol[119075|ROOT|48fe494b-ee02-4f08-b7aa-975baecd7b3e|21474836480]"
at com.google.gson.JsonObjectDeserializationVisitor.visitFieldUsingCustomHandler(JsonObjectDeserializationVisitor.java:100)
at com.google.gson.ReflectingFieldNavigator.visitFieldsReflectively(ReflectingFieldNavigator.java:63)
at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:120)
at com.google.gson.JsonDeserializationContextDefault.fromJsonPrimitive(JsonDeserializationContextDefault.java:85)
at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:56)
at com.google.gson.MapTypeAdapter.deserialize(MapTypeAdapter.java:67)
at com.google.gson.MapTypeAdapter.deserialize(MapTypeAdapter.java:33)
at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:51)
at com.google.gson.JsonDeserializationVisitor.invokeCustomDeserializer(JsonDeserializationVisitor.java:92)
at com.google.gson.JsonObjectDeserializationVisitor.visitFieldUsingCustomHandler(JsonObjectDeserializationVisitor.java:117)
at com.google.gson.ReflectingFieldNavigator.visitFieldsReflectively(ReflectingFieldNavigator.java:63)
at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:120)
at com.google.gson.JsonDeserializationContextDefault.fromJsonObject(JsonDeserializationContextDefault.java:76)
at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:54)
at com.google.gson.Gson.fromJson(Gson.java:551)
at com.google.gson.Gson.fromJson(Gson.java:521)
at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:80)
at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:40)
at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:51)
at com.google.gson.JsonDeserializationVisitor.invokeCustomDeserializer(JsonDeserializationVisitor.java:92)
at com.google.gson.JsonDeserializationVisitor.visitUsingCustomHandler(JsonDeserializationVisitor.java:80)
at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:101)
at com.google.gson.JsonDeserializationContextDefault.fromJsonArray(JsonDeserializationContextDefault.java:67)
at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:52)
at com.google.gson.Gson.fromJson(Gson.java:551)
at com.google.gson.Gson.fromJson(Gson.java:498)
at com.cloud.agent.transport.Request.getCommands(Request.java:250)
at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:346)
at com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:666)
at com.cloud.utils.nio.Task.run(Task.java:84)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
WARN [c.c.u.n.Task] (AgentManager-Handler-11:null) Caught the following exception but pushing on
com.google.gson.JsonParseException: Expecting object found: "Vol[119075|ROOT|48fe494b-ee02-4f08-b7aa-975baecd7b3e|21474836480]"
at com.google.gson.JsonObjectDeserializationVisitor.visitFieldUsingCustomHandler(JsonObjectDeserializationVisitor.java:100)
at com.google.gson.ReflectingFieldNavigator.visitFieldsReflectively(ReflectingFieldNavigator.java:63)
at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:120)
at com.google.gson.JsonDeserializationContextDefault.fromJsonPrimitive(JsonDeserializationContextDefault.java:85)
at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:56)
at com.google.gson.MapTypeAdapter.deserialize(MapTypeAdapter.java:67)
at com.google.gson.MapTypeAdapter.deserialize(MapTypeAdapter.java:33)
at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:51)
at com.google.gson.JsonDeserializationVisitor.invokeCustomDeserializer(JsonDeserializationVisitor.java:92)
at com.google.gson.JsonObjectDeserializationVisitor.visitFieldUsingCustomHandler(JsonObjectDeserializationVisitor.java:117)
at com.google.gson.ReflectingFieldNavigator.visitFieldsReflectively(ReflectingFieldNavigator.java:63)
at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:120)
at com.google.gson.JsonDeserializationContextDefault.fromJsonObject(JsonDeserializationContextDefault.java:76)
at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:54)
at com.google.gson.Gson.fromJson(Gson.java:551)
at com.google.gson.Gson.fromJson(Gson.java:521)
at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:80)
at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:40)
at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:51)
at com.google.gson.JsonDeserializationVisitor.invokeCustomDeserializer(JsonDeserializationVisitor.java:92)
at com.google.gson.JsonDeserializationVisitor.visitUsingCustomHandler(JsonDeserializationVisitor.java:80)
at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:101)
at com.google.gson.JsonDeserializationContextDefault.fromJsonArray(JsonDeserializationContextDefault.java:67)
at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:52)
at com.google.gson.Gson.fromJson(Gson.java:551)
at com.google.gson.Gson.fromJson(Gson.java:498)
at com.cloud.agent.transport.Request.getCommands(Request.java:250)
at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:346)
at com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:666)
at com.cloud.utils.nio.Task.run(Task.java:84)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
When trying the migration with a single management server running, no issues are seen.
