Uploaded image for project: 'CloudStack'
  1. CloudStack
  2. CLOUDSTACK-3774

[XenServer]NPE while creating template from snapshot on a upgraded setup

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 4.2.0
    • Fix Version/s: 4.2.0
    • Security Level: Public (Anyone can view this level - this is the default.)
    • Labels:
      None

      Description

      Setup: Xen 6.1 hosts , Two zone ( Zone1 with local storage ,Zone 2 - NFS )

      Upgraded from 3.0.6 to 4.2

      Steps:

      1) Create Account X and deploy VM on a NFS storage Zone 3
      2) Create snapshot from ROOT volume of this instance
      3) Tried to create template from this snapshot.

      Observations :

      1. It failed with NPE.
      2. UI results NPE as a failure message

      2013-07-24 19:07:50,164 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots already exists on secondary storage, and is mounted at /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
      2013-07-24 19:07:50,164 INFO [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots directory created/exists on Secondary Storage.
      2013-07-24 19:07:50,165 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes already exists on secondary storage, and is mounted at /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
      2013-07-24 19:07:50,166 INFO [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes directory created/exists on Secondary Storage.
      2013-07-24 19:07:50,166 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Executing: sudo mount
      2013-07-24 19:07:50,224 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Execution is successful.
      2013-07-24 19:07:50,224 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) mount point /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1 already exists
      2013-07-24 19:07:50,309 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Executing: null -p /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40 -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
      2013-07-24 19:07:50,309 WARN [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Exception: null -p /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40 -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
      java.lang.NullPointerException
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
      at com.cloud.utils.script.Script.execute(Script.java:183)
      at com.cloud.utils.script.Script.execute(Script.java:161)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
      at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
      at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
      at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
      at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
      at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
      at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
      at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
      at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:679)
      2013-07-24 19:07:50,348 DEBUG [cloud.template.TemplateManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Failed to create templatejava.lang.NullPointerException
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
      at com.cloud.utils.script.Script.execute(Script.java:183)
      at com.cloud.utils.script.Script.execute(Script.java:161)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
      at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
      at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
      at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
      at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
      at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
      at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
      at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
      at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
      at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:679)

      2013-07-24 19:07:50,381 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Unexpected exception while executing org.apache.cloudstack.api.command.user.template.CreateTemplateCmd
      com.cloud.utils.exception.CloudRuntimeException: Failed to create templatejava.lang.NullPointerException
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
      at com.cloud.utils.script.Script.execute(Script.java:183)
      at com.cloud.utils.script.Script.execute(Script.java:161)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
      at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
      at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
      at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
      at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
      at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
      at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
      at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
      at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
      at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:679)

      at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1369)
      at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
      at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
      at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
      at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:679)
      2013-07-24 19:07:50,390 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Complete async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ], jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to create templatejava.lang.NullPointerException
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
      at com.cloud.utils.script.Script.execute(Script.java:183)
      at com.cloud.utils.script.Script.execute(Script.java:161)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
      at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
      at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
      at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
      at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
      at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
      at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
      at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
      at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
      at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:679)

      2013-07-24 19:07:52,603 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===START=== 10.144.6.19 – GET command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
      2013-07-24 19:07:52,629 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-15:null) Async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ] completed
      2013-07-24 19:07:52,640 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===END=== 10.144.6.19 – GET command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
      2013-07-24 19:07:52,769 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-11:null) SeqA 10-8916: Processing Seq 10-8916: { Cmd , MgmtId: -1, via: 10, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":18,"_loadInfo":"

      {\n \"connections\": []\n}

      ","wait":0}}] }

        Attachments

        1. sysvm.log
          0.3 kB
          Sailaja Mada
        2. cloud.log
          1.0 kB
          Sailaja Mada
        3. postupgradecloud-backup.dmp
          797 kB
          Sailaja Mada
        4. management-server.log
          4.41 MB
          Sailaja Mada
        5. apilog.log
          1.38 MB
          Sailaja Mada
        6. snap.png
          340 kB
          Sailaja Mada
        7. templatefromsnapshot.png
          453 kB
          Sailaja Mada
        8. management-server.log
          7.70 MB
          Sailaja Mada
        9. beforeupgrade_mysqldump.dmp
          507 kB
          Sailaja Mada
        10. apilog.log
          1.79 MB
          Sailaja Mada
        11. afterupgrademysqldumpusage1.dmp
          908 kB
          Sailaja Mada

          Activity

            People

            • Assignee:
              devdeep Devdeep Singh
              Reporter:
              sailaja Sailaja Mada
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: