Uploaded image for project: 'CloudStack'
  1. CloudStack
  2. CLOUDSTACK-8937

Xenserver - VM migration with storage fails in a clustered management server setup

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.5.2
    • Fix Version/s: None
    • Component/s: None
    • Security Level: Public (Anyone can view this level - this is the default.)
    • Labels:
      None

      Description

      When using the migrateVirtualMachineWithVolume, in a clustered management server environment, migrations intermittently fail. This appears to be a similar issue to this jira ticket:

      https://issues.apache.org/jira/browse/CLOUDSTACK-8412

      For reference, the error is:

      ERROR [c.c.a.t.Request] (AgentManager-Handler-11:null) Caught problem with [{"com.cloud.agent.api.MigrateWithStorageCommand":{"vm":{"id":26631,"name":"i-2-26631-VM","bootloader":"PyGrub","type":"Us
      er","cpus":1,"minSpeed":1200,"maxSpeed":1200,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"CentOS 5.6 (64-bit)","platformEmulator":"CentOS 5 (64-bit)","bootArgs":"","enableHA":false,"
      limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"","params":{"memoryOvercommitRatio":"1","platform":"viridian:true;acpi:1;apic:true;pae:true;nx:t
      rue","hypervisortoolsversion":"xenserver56","cpuOvercommitRatio":"4"},"uuid":"b37e76e8-e19a-4100-bf1c-1ee8eba674f2","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"8f51
      e73f-9bfc-4a37-bf86-664067a2cf40","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"b114d58e-a37a-6433-c0ff-07f3613a3c68","id":219,"poolType":"LVM","ho
      st":"192.168.97.2","path":"lvm","port":0,"url":"LVM://192.168.97.2/lvm/?ROLE\u003dPrimary\u0026STOREUUID\u003db114d58e-a37a-6433-c0ff-07f3613a3c68"}},"name":"ROOT-26631","size":21474836480,"path":"48
      fe494b-ee02-4f08-b7aa-975baecd7b3e","volumeId":119075,"vmName":"i-2-26631-VM","accountId":2,"format":"VHD","provisioningType":"THIN","id":119075,"deviceId":0,"cacheMode":"NONE","hypervisorType":"Xe
      nServer"}},"diskSeq":0,"path":"48fe494b-ee02-4f08-b7aa-975baecd7b3e","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"192.168.97.2","volumeSize":"21474836480"}},{"data":{
      "org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":2000,"defaultNic":true,"pxe
      Disable":false,"nicUuid":"aaef8c14-784c-403f-91fc-b6a3167f2595","uuid":"508d0384-5bf0-4434-8155-0b5bb1b33b43","ip":"192.168.8.112","netmask":"255.255.254.0","gateway":"192.168.9.254","mac":"06:68:80:
      00:00:7f","dns1":"192.168.0.224","dns2":"192.168.0.224","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true,"name":"CS-Guest-Public"}],"vcpuMaxLimit":
      16},"volumeToFiler":{"Vol[119075|ROOT|48fe494b-ee02-4f08-b7aa-975baecd7b3e|21474836480]":{"id":223,"uuid":"bc01ca2b-e6d8-a856-27b5-6f6b55a06b1d","host":"192.168.97.1","path":"lvm","port":0,"type":"L
      VM"}},"contextMap":{"job":"job-347068/job-347069"},"wait":0}}]
      com.google.gson.JsonParseException: Expecting object found: "Vol[119075|ROOT|48fe494b-ee02-4f08-b7aa-975baecd7b3e|21474836480]"
              at com.google.gson.JsonObjectDeserializationVisitor.visitFieldUsingCustomHandler(JsonObjectDeserializationVisitor.java:100)
              at com.google.gson.ReflectingFieldNavigator.visitFieldsReflectively(ReflectingFieldNavigator.java:63)
              at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:120)
              at com.google.gson.JsonDeserializationContextDefault.fromJsonPrimitive(JsonDeserializationContextDefault.java:85)
              at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:56)
              at com.google.gson.MapTypeAdapter.deserialize(MapTypeAdapter.java:67)
              at com.google.gson.MapTypeAdapter.deserialize(MapTypeAdapter.java:33)
              at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:51)
              at com.google.gson.JsonDeserializationVisitor.invokeCustomDeserializer(JsonDeserializationVisitor.java:92)
              at com.google.gson.JsonObjectDeserializationVisitor.visitFieldUsingCustomHandler(JsonObjectDeserializationVisitor.java:117)
              at com.google.gson.ReflectingFieldNavigator.visitFieldsReflectively(ReflectingFieldNavigator.java:63)
              at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:120)
              at com.google.gson.JsonDeserializationContextDefault.fromJsonObject(JsonDeserializationContextDefault.java:76)
              at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:54)
              at com.google.gson.Gson.fromJson(Gson.java:551)
              at com.google.gson.Gson.fromJson(Gson.java:521)
              at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:80)
              at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:40)
              at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:51)
              at com.google.gson.JsonDeserializationVisitor.invokeCustomDeserializer(JsonDeserializationVisitor.java:92)
              at com.google.gson.JsonDeserializationVisitor.visitUsingCustomHandler(JsonDeserializationVisitor.java:80)
              at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:101)
              at com.google.gson.JsonDeserializationContextDefault.fromJsonArray(JsonDeserializationContextDefault.java:67)
              at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:52)
              at com.google.gson.Gson.fromJson(Gson.java:551)
              at com.google.gson.Gson.fromJson(Gson.java:498)
              at com.cloud.agent.transport.Request.getCommands(Request.java:250)
              at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:346)
              at com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:666)
              at com.cloud.utils.nio.Task.run(Task.java:84)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
              at java.lang.Thread.run(Thread.java:722)
      WARN  [c.c.u.n.Task] (AgentManager-Handler-11:null) Caught the following exception but pushing on
      com.google.gson.JsonParseException: Expecting object found: "Vol[119075|ROOT|48fe494b-ee02-4f08-b7aa-975baecd7b3e|21474836480]"
              at com.google.gson.JsonObjectDeserializationVisitor.visitFieldUsingCustomHandler(JsonObjectDeserializationVisitor.java:100)
              at com.google.gson.ReflectingFieldNavigator.visitFieldsReflectively(ReflectingFieldNavigator.java:63)
              at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:120)
              at com.google.gson.JsonDeserializationContextDefault.fromJsonPrimitive(JsonDeserializationContextDefault.java:85)
              at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:56)
              at com.google.gson.MapTypeAdapter.deserialize(MapTypeAdapter.java:67)
              at com.google.gson.MapTypeAdapter.deserialize(MapTypeAdapter.java:33)
              at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:51)
              at com.google.gson.JsonDeserializationVisitor.invokeCustomDeserializer(JsonDeserializationVisitor.java:92)
              at com.google.gson.JsonObjectDeserializationVisitor.visitFieldUsingCustomHandler(JsonObjectDeserializationVisitor.java:117)
              at com.google.gson.ReflectingFieldNavigator.visitFieldsReflectively(ReflectingFieldNavigator.java:63)
              at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:120)
              at com.google.gson.JsonDeserializationContextDefault.fromJsonObject(JsonDeserializationContextDefault.java:76)
              at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:54)
              at com.google.gson.Gson.fromJson(Gson.java:551)
              at com.google.gson.Gson.fromJson(Gson.java:521)
              at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:80)
              at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:40)
              at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:51)
              at com.google.gson.JsonDeserializationVisitor.invokeCustomDeserializer(JsonDeserializationVisitor.java:92)
              at com.google.gson.JsonDeserializationVisitor.visitUsingCustomHandler(JsonDeserializationVisitor.java:80)
              at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:101)
              at com.google.gson.JsonDeserializationContextDefault.fromJsonArray(JsonDeserializationContextDefault.java:67)
              at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:52)
              at com.google.gson.Gson.fromJson(Gson.java:551)
              at com.google.gson.Gson.fromJson(Gson.java:498)
              at com.cloud.agent.transport.Request.getCommands(Request.java:250)
              at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:346)
              at com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:666)
              at com.cloud.utils.nio.Task.run(Task.java:84)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
              at java.lang.Thread.run(Thread.java:722)
      

      When trying the migration with a single management server running, no issues are seen.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              acldstkusr CS User
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: