[CLOUDSTACK-8768] [HyperV]Migrating volume from cluster wide storage to Zone wide storage or viceversa is failing.

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 4.6.0
• Security Level: Public (Anyone can view this level - this is the default.)
• Labels: hyper-v
• Assignee: Unassigned
• Reporter: Deepthi Machiraju
• Created: 25/Aug/15 12:12
• Updated: 26/Aug/15 04:21

Description

• Upload a volume of type VHD.
• Attach it to an instance and then detach.
• if the volume is residing on cluster store , migrate to zone storage. If volume resides on Zone storage migrate to cluster storage.

Observation:

Migrate of volume is failing with following reason " Resource [StoragePool:1] is unreachable: Migrate volume failed: Local template file missing from \\10.81.28.134\storage\zwps1\22bcb969-9745-44b4-84ee-241014fa7aed.vhd "

2015-08-25 11:57:07,862 DEBUG [c.c.a.t.Request] (API-Job-Executor-55:ctx-570d388e job-127 ctx-ac7e0402) Seq 3-8521091969961690678: Received: { Ans: , MgmtId: 227345751881375, via: 3, Ver: v1, Flags: 10,

{ Answer }

}
2015-08-25 11:57:07,863 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl] (API-Job-Executor-55:ctx-570d388e job-127 ctx-ac7e0402) Volume 26 is not found on image store 1, so no need to delete
2015-08-25 11:57:07,875 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl] (API-Job-Executor-55:ctx-570d388e job-127 ctx-ac7e0402) Unsupported data object (VOLUME, org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@590977cc), no need to delete from object in store ref table
2015-08-25 11:57:07,903 DEBUG [o.a.c.s.v.VolumeServiceImpl] (API-Job-Executor-55:ctx-570d388e job-127 ctx-ac7e0402) Marking volume that was never created as destroyed: Vol[27|vm=null|DATADISK]
2015-08-25 11:57:07,919 ERROR [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-55:ctx-570d388e job-127 ctx-ac7e0402) Migrate volume failed:Local template file missing from \\10.81.28.134\storage\zwps1\22bcb969-9745-44b4-84ee-241014fa7aed.vhd
2015-08-25 11:57:07,919 DEBUG [c.c.s.VolumeApiServiceImpl] (API-Job-Executor-55:ctx-570d388e job-127 ctx-ac7e0402) Failed to migrate volume
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Migrate volume failed: Local template file missing from \\10.81.28.134\storage\zwps1\22bcb969-9745-44b4-84ee-241014fa7aed.vhd
at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolume(VolumeOrchestrator.java:942)
at com.cloud.storage.VolumeApiServiceImpl.orchestrateMigrateVolume(VolumeApiServiceImpl.java:1965)
at com.cloud.storage.VolumeApiServiceImpl.migrateVolume(VolumeApiServiceImpl.java:1951)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy195.migrateVolume(Unknown Source)
at org.apache.cloudstack.api.command.admin.volume.MigrateVolumeCmdByAdmin.execute(MigrateVolumeCmdByAdmin.java:38)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:537)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:494)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2015-08-25 11:57:07,928 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-55:ctx-570d388e job-127) Unexpected exception while executing org.apache.cloudstack.api.command.admin.volume.MigrateVolumeCmdByAdmin
com.cloud.utils.exception.CloudRuntimeException: Resource [StoragePool:1] is unreachable: Migrate volume failed: Local template file missing from \\10.81.28.134\storage\zwps1\22bcb969-9745-44b4-84ee-241014fa7aed.vhd
at com.cloud.storage.VolumeApiServiceImpl.orchestrateMigrateVolume(VolumeApiServiceImpl.java:1969)
at com.cloud.storage.VolumeApiServiceImpl.migrateVolume(VolumeApiServiceImpl.java:1951)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy195.migrateVolume(Unknown Source)
at org.apache.cloudstack.api.command.admin.volume.MigrateVolumeCmdByAdmin.execute(MigrateVolumeCmdByAdmin.java:38)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:537)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:494)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2015-08-25 11:57:07,929 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-55:ctx-570d388e job-127) Complete async job-127, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/

{"uuidList":[],"errorcode":530,"errortext":"Resource [StoragePool:1] is unreachable: Migrate volume failed: Local template file missing from \\\\10.81.28.134\\storage\\zwps1\\22bcb969-9745-44b4-84ee-241014fa7aed.vhd"}

Expected Result:

Migrate should be sucessful.

Attached MS log and DB dump.