CloudStack
  1. CloudStack
  2. CLOUDSTACK-3774

[XenServer]NPE while creating template from snapshot on a upgraded setup

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 4.2.0
    • Fix Version/s: 4.2.0
    • Security Level: Public (Anyone can view this level - this is the default.)
    • Labels:
      None

      Description

      Setup: Xen 6.1 hosts , Two zone ( Zone1 with local storage ,Zone 2 - NFS )

      Upgraded from 3.0.6 to 4.2

      Steps:

      1) Create Account X and deploy VM on a NFS storage Zone 3
      2) Create snapshot from ROOT volume of this instance
      3) Tried to create template from this snapshot.

      Observations :

      1. It failed with NPE.
      2. UI results NPE as a failure message

      2013-07-24 19:07:50,164 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots already exists on secondary storage, and is mounted at /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
      2013-07-24 19:07:50,164 INFO [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots directory created/exists on Secondary Storage.
      2013-07-24 19:07:50,165 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes already exists on secondary storage, and is mounted at /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
      2013-07-24 19:07:50,166 INFO [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes directory created/exists on Secondary Storage.
      2013-07-24 19:07:50,166 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Executing: sudo mount
      2013-07-24 19:07:50,224 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Execution is successful.
      2013-07-24 19:07:50,224 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) mount point /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1 already exists
      2013-07-24 19:07:50,309 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Executing: null -p /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40 -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
      2013-07-24 19:07:50,309 WARN [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Exception: null -p /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40 -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
      java.lang.NullPointerException
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
      at com.cloud.utils.script.Script.execute(Script.java:183)
      at com.cloud.utils.script.Script.execute(Script.java:161)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
      at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
      at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
      at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
      at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
      at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
      at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
      at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
      at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:679)
      2013-07-24 19:07:50,348 DEBUG [cloud.template.TemplateManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Failed to create templatejava.lang.NullPointerException
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
      at com.cloud.utils.script.Script.execute(Script.java:183)
      at com.cloud.utils.script.Script.execute(Script.java:161)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
      at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
      at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
      at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
      at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
      at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
      at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
      at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
      at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
      at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:679)

      2013-07-24 19:07:50,381 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Unexpected exception while executing org.apache.cloudstack.api.command.user.template.CreateTemplateCmd
      com.cloud.utils.exception.CloudRuntimeException: Failed to create templatejava.lang.NullPointerException
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
      at com.cloud.utils.script.Script.execute(Script.java:183)
      at com.cloud.utils.script.Script.execute(Script.java:161)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
      at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
      at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
      at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
      at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
      at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
      at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
      at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
      at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
      at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:679)

      at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1369)
      at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
      at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
      at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
      at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:679)
      2013-07-24 19:07:50,390 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Complete async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ], jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to create templatejava.lang.NullPointerException
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
      at com.cloud.utils.script.Script.execute(Script.java:183)
      at com.cloud.utils.script.Script.execute(Script.java:161)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
      at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
      at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
      at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
      at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
      at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
      at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
      at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
      at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
      at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
      at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
      at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
      at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
      at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:679)

      2013-07-24 19:07:52,603 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===START=== 10.144.6.19 – GET command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
      2013-07-24 19:07:52,629 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-15:null) Async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ] completed
      2013-07-24 19:07:52,640 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===END=== 10.144.6.19 – GET command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
      2013-07-24 19:07:52,769 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-11:null) SeqA 10-8916: Processing Seq 10-8916: { Cmd , MgmtId: -1, via: 10, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":18,"_loadInfo":"

      {\n \"connections\": []\n}

      ","wait":0}}] }

      1. sysvm.log
        0.3 kB
        Sailaja Mada
      2. cloud.log
        1.0 kB
        Sailaja Mada
      3. postupgradecloud-backup.dmp
        797 kB
        Sailaja Mada
      4. management-server.log
        4.41 MB
        Sailaja Mada
      5. apilog.log
        1.38 MB
        Sailaja Mada
      6. snap.png
        340 kB
        Sailaja Mada
      7. templatefromsnapshot.png
        453 kB
        Sailaja Mada
      8. management-server.log
        7.70 MB
        Sailaja Mada
      9. beforeupgrade_mysqldump.dmp
        507 kB
        Sailaja Mada
      10. apilog.log
        1.79 MB
        Sailaja Mada
      11. afterupgrademysqldumpusage1.dmp
        908 kB
        Sailaja Mada

        Activity

        Sailaja Mada created issue -
        Sailaja Mada made changes -
        Field Original Value New Value
        Attachment afterupgrademysqldumpusage1.dmp [ 12593888 ]
        Attachment apilog.log [ 12593889 ]
        Attachment beforeupgrade_mysqldump.dmp [ 12593890 ]
        Sailaja Mada made changes -
        Attachment management-server.log [ 12593892 ]
        Sailaja Mada made changes -
        Description Setup: Xen 6.1 hosts , Two zone ( Zone1 with local storage ,Zone 2 - NFS )

        Upgraded from 3.0.6 to 4.2

        Steps:

        1) Create Account X and deploy VM on a NFS storage Zone 3
        2) Create snapshot from ROOT volume of this instance
        3) Tried to create template from this snapshot.

        It failed with NPE:
        2013-07-24 19:07:50,164 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots already exists on secondary storage, and is mounted at /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
        2013-07-24 19:07:50,164 INFO [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots directory created/exists on Secondary Storage.
        2013-07-24 19:07:50,165 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes already exists on secondary storage, and is mounted at /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
        2013-07-24 19:07:50,166 INFO [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes directory created/exists on Secondary Storage.
        2013-07-24 19:07:50,166 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Executing: sudo mount
        2013-07-24 19:07:50,224 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Execution is successful.
        2013-07-24 19:07:50,224 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) mount point /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1 already exists
        2013-07-24 19:07:50,309 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Executing: null -p /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40 -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
        2013-07-24 19:07:50,309 WARN [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Exception: null -p /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40 -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
        java.lang.NullPointerException
                at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
                at com.cloud.utils.script.Script.execute(Script.java:183)
                at com.cloud.utils.script.Script.execute(Script.java:161)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
                at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
                at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
                at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
                at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
                at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
                at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
                at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
                at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
                at java.util.concurrent.FutureTask.run(FutureTask.java:166)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
                at java.lang.Thread.run(Thread.java:679)
        2013-07-24 19:07:50,348 DEBUG [cloud.template.TemplateManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Failed to create templatejava.lang.NullPointerException
                at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
                at com.cloud.utils.script.Script.execute(Script.java:183)
                at com.cloud.utils.script.Script.execute(Script.java:161)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
                at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
                at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
                at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
                at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
                at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
                at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
                at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
                at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
                at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
                at java.util.concurrent.FutureTask.run(FutureTask.java:166)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
                at java.lang.Thread.run(Thread.java:679)

        2013-07-24 19:07:50,381 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Unexpected exception while executing org.apache.cloudstack.api.command.user.template.CreateTemplateCmd
        com.cloud.utils.exception.CloudRuntimeException: Failed to create templatejava.lang.NullPointerException
                at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
                at com.cloud.utils.script.Script.execute(Script.java:183)
                at com.cloud.utils.script.Script.execute(Script.java:161)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
                at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
                at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
                at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
                at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
                at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
                at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
                at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
                at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
                at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
                at java.util.concurrent.FutureTask.run(FutureTask.java:166)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
         at java.lang.Thread.run(Thread.java:679)

                at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1369)
                at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
                at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
                at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
                at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
                at java.util.concurrent.FutureTask.run(FutureTask.java:166)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
                at java.lang.Thread.run(Thread.java:679)
        2013-07-24 19:07:50,390 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Complete async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ], jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to create templatejava.lang.NullPointerException
                at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
                at com.cloud.utils.script.Script.execute(Script.java:183)
                at com.cloud.utils.script.Script.execute(Script.java:161)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
                at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
                at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
                at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
                at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
                at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
                at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
                at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
                at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
                at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
                at java.util.concurrent.FutureTask.run(FutureTask.java:166)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
                at java.lang.Thread.run(Thread.java:679)

        2013-07-24 19:07:52,603 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===START=== 10.144.6.19 -- GET command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
        2013-07-24 19:07:52,629 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-15:null) Async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ] completed
        2013-07-24 19:07:52,640 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===END=== 10.144.6.19 -- GET command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
        2013-07-24 19:07:52,769 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-11:null) SeqA 10-8916: Processing Seq 10-8916: { Cmd , MgmtId: -1, via: 10, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":18,"_loadInfo":"{\n \"connections\": []\n}","wait":0}}] }
        Setup: Xen 6.1 hosts , Two zone ( Zone1 with local storage ,Zone 2 - NFS )

        Upgraded from 3.0.6 to 4.2

        Steps:

        1) Create Account X and deploy VM on a NFS storage Zone 3
        2) Create snapshot from ROOT volume of this instance
        3) Tried to create template from this snapshot.

        Observations :

        1. It failed with NPE.
        2. UI results NPE as a failure message


        2013-07-24 19:07:50,164 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots already exists on secondary storage, and is mounted at /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
        2013-07-24 19:07:50,164 INFO [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots directory created/exists on Secondary Storage.
        2013-07-24 19:07:50,165 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes already exists on secondary storage, and is mounted at /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
        2013-07-24 19:07:50,166 INFO [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes directory created/exists on Secondary Storage.
        2013-07-24 19:07:50,166 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Executing: sudo mount
        2013-07-24 19:07:50,224 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Execution is successful.
        2013-07-24 19:07:50,224 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) mount point /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1 already exists
        2013-07-24 19:07:50,309 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Executing: null -p /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40 -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
        2013-07-24 19:07:50,309 WARN [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Exception: null -p /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40 -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
        java.lang.NullPointerException
                at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
                at com.cloud.utils.script.Script.execute(Script.java:183)
                at com.cloud.utils.script.Script.execute(Script.java:161)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
                at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
                at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
                at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
                at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
                at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
                at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
                at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
                at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
                at java.util.concurrent.FutureTask.run(FutureTask.java:166)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
                at java.lang.Thread.run(Thread.java:679)
        2013-07-24 19:07:50,348 DEBUG [cloud.template.TemplateManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Failed to create templatejava.lang.NullPointerException
                at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
                at com.cloud.utils.script.Script.execute(Script.java:183)
                at com.cloud.utils.script.Script.execute(Script.java:161)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
                at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
                at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
                at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
                at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
                at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
                at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
                at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
                at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
                at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
                at java.util.concurrent.FutureTask.run(FutureTask.java:166)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
                at java.lang.Thread.run(Thread.java:679)

        2013-07-24 19:07:50,381 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Unexpected exception while executing org.apache.cloudstack.api.command.user.template.CreateTemplateCmd
        com.cloud.utils.exception.CloudRuntimeException: Failed to create templatejava.lang.NullPointerException
                at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
                at com.cloud.utils.script.Script.execute(Script.java:183)
                at com.cloud.utils.script.Script.execute(Script.java:161)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
                at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
                at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
                at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
                at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
                at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
                at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
                at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
                at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
                at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
                at java.util.concurrent.FutureTask.run(FutureTask.java:166)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
         at java.lang.Thread.run(Thread.java:679)

                at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1369)
                at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
                at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
                at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
                at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
                at java.util.concurrent.FutureTask.run(FutureTask.java:166)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
                at java.lang.Thread.run(Thread.java:679)
        2013-07-24 19:07:50,390 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Complete async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ], jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to create templatejava.lang.NullPointerException
                at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
                at com.cloud.utils.script.Script.execute(Script.java:183)
                at com.cloud.utils.script.Script.execute(Script.java:161)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
                at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
                at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
                at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
                at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
                at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
                at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
                at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
                at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
                at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
                at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
                at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
                at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
                at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
                at java.util.concurrent.FutureTask.run(FutureTask.java:166)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
                at java.lang.Thread.run(Thread.java:679)

        2013-07-24 19:07:52,603 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===START=== 10.144.6.19 -- GET command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
        2013-07-24 19:07:52,629 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-15:null) Async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ] completed
        2013-07-24 19:07:52,640 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===END=== 10.144.6.19 -- GET command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
        2013-07-24 19:07:52,769 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-11:null) SeqA 10-8916: Processing Seq 10-8916: { Cmd , MgmtId: -1, via: 10, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":18,"_loadInfo":"{\n \"connections\": []\n}","wait":0}}] }
        Sailaja Mada made changes -
        Attachment templatefromsnapshot.png [ 12593893 ]
        Sailaja Mada made changes -
        Fix Version/s 4.2.0 [ 12323906 ]
        Hide
        Devdeep Singh added a comment -

        Can you look into the secondary storage systemvm and see if a script create_privatetemplate_from_snapshot_xen.sh is present in it or not? This is probably a new script that was added and we need to check if it is present in the upgraded setup.

        Show
        Devdeep Singh added a comment - Can you look into the secondary storage systemvm and see if a script create_privatetemplate_from_snapshot_xen.sh is present in it or not? This is probably a new script that was added and we need to check if it is present in the upgraded setup.
        Hide
        Sailaja Mada added a comment -

        This script is available @ ssvm. I have attached the snap for the same.

        Show
        Sailaja Mada added a comment - This script is available @ ssvm. I have attached the snap for the same.
        Sailaja Mada made changes -
        Attachment snap.png [ 12593940 ]
        Ram Ganesh made changes -
        Assignee Devdeep Singh [ devdeep ]
        Hide
        Devdeep Singh added a comment -

        Tried it against an upgraded setup and we weren't able to reproduce the issue. The setup against which the issue was tried was upgraded from 2.2.15 to 4.2 (going through 3.0.6 in between). We did observe an issue in creating template from snapshot, but it was because vhd-util wasn't present in the updated 64 bit system vm. vhd-util not being present issue has already been fixed in another commit.

        Looking at the logs it seems that the SSVM was in alert state from cloudstack. I also observe that the LocalNfsSecondaryStorageResource getting picked up. From the code it looks like this happens when we are not able to find any up and enabled SSVMs.

        Please reopen the issue if you observe it again.

        Show
        Devdeep Singh added a comment - Tried it against an upgraded setup and we weren't able to reproduce the issue. The setup against which the issue was tried was upgraded from 2.2.15 to 4.2 (going through 3.0.6 in between). We did observe an issue in creating template from snapshot, but it was because vhd-util wasn't present in the updated 64 bit system vm. vhd-util not being present issue has already been fixed in another commit. Looking at the logs it seems that the SSVM was in alert state from cloudstack. I also observe that the LocalNfsSecondaryStorageResource getting picked up. From the code it looks like this happens when we are not able to find any up and enabled SSVMs. Please reopen the issue if you observe it again.
        Devdeep Singh made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Cannot Reproduce [ 5 ]
        Hide
        Sailaja Mada added a comment -

        This issue is reproduced on a upgraded setup with 4.2 latest builds. Hence reopening the bug.

        Show
        Sailaja Mada added a comment - This issue is reproduced on a upgraded setup with 4.2 latest builds. Hence reopening the bug.
        Sailaja Mada made changes -
        Resolution Cannot Reproduce [ 5 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Sailaja Mada made changes -
        Priority Critical [ 2 ] Blocker [ 1 ]
        Hide
        Sailaja Mada added a comment -

        2013-08-05 17:34:57,223 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-19:null) submit async job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ], details: AsyncJobVO {id:80, userId: 3, accountId: 3, sessionKey: null, instanceType: Template, instanceId: 208, cmd: org.apache.cloudstack.api.command.user.template.CreateTemplateCmd, cmdOriginator: null, cmdInfo:

        {"sessionkey":"kS+6Kqk1tmAqe2ZaWxuYGCHGkzY\u003d","cmdEventType":"TEMPLATE.CREATE","ctxUserId":"3","httpmethod":"GET","osTypeId":"142","isPublic":"true","isdynamicallyscalable":"false","response":"json","id":"208","displayText":"tmpl1","snapshotid":"bb36997c-0e86-499b-9013-3454a881b43c","passwordEnabled":"false","name":"tmpl1","_":"1375704520095","ctxAccountId":"3","ctxStartEventId":"317"}

        , cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, processStatus: 0, resultCode: 0, result: null, initMsid: 7674049379768, completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
        2013-08-05 17:34:57,226 DEBUG [cloud.api.ApiServlet] (catalina-exec-19:null) ===END=== 10.144.6.19 – GET command=createTemplate&response=json&sessionkey=kS%2B6Kqk1tmAqe2ZaWxuYGCHGkzY%3D&snapshotid=bb36997c-0e86-499b-9013-3454a881b43c&name=tmpl1&displayText=tmpl1&osTypeId=142&isPublic=true&passwordEnabled=false&isdynamicallyscalable=false&_=1375704520095
        2013-08-05 17:34:57,229 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Executing org.apache.cloudstack.api.command.user.template.CreateTemplateCmd for job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]
        2013-08-05 17:34:57,245 DEBUG [cloud.user.AccountManagerImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Access to Acct[3-user1] granted to Acct[3-user1] by DomainChecker_EnhancerByCloudStack_d17a7327
        2013-08-05 17:34:57,283 DEBUG [storage.image.TemplateDataFactoryImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) template 208 is already in store:2, type:Image
        2013-08-05 17:34:57,297 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) copyAsync inspecting src type SNAPSHOT copyAsync inspecting dest type TEMPLATE
        2013-08-05 17:34:57,331 INFO [storage.endpoint.DefaultEndPointSelector] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) No running ssvm is found, so command will be sent to LocalHostEndPoint
        2013-08-05 17:34:57,362 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Unable to create mount point: /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476
        2013-08-05 17:34:57,363 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Unable to create mount point: /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476
        2013-08-05 17:34:57,365 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Executing: null -p /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476/snapshots/3/16 -s 3c4326d9-bf4e-45cf-81d4-7024f8efedd5.vhd -n bdb99a9e-94d7-43e1-9f00-beaa77ad7748.vhd -t /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476/template/tmpl/3/208
        2013-08-05 17:34:57,365 WARN [storage.resource.NfsSecondaryStorageResource] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Exception: null -p /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476/snapshots/3/16 -s 3c4326d9-bf4e-45cf-81d4-7024f8efedd5.vhd -n bdb99a9e-94d7-43e1-9f00-beaa77ad7748.vhd -t /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476/template/tmpl/3/208
        java.lang.NullPointerException
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
        at com.cloud.utils.script.Script.execute(Script.java:183)
        at com.cloud.utils.script.Script.execute(Script.java:161)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:363)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:404)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:482)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:532)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:200)
        at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
        at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:416)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:371)
        at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
        at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
        at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
        at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
        2013-08-05 17:34:57,386 DEBUG [cloud.template.TemplateManagerImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Failed to create templatejava.lang.NullPointerException
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
        at com.cloud.utils.script.Script.execute(Script.java:183)
        at com.cloud.utils.script.Script.execute(Script.java:161)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:363)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:404)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:482)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:532)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:200)
        at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
        at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:416)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:371)
        at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
        at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
        at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
        at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
        at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.CGLIB$createPrivateTemplate$32(<generated>)
        at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095_FastClassByCloudStack_ea3ab7db.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.createPrivateTemplate(<generated>)
        at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)

        2013-08-05 17:34:57,408 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Unexpected exception while executing org.apache.cloudstack.api.command.user.template.CreateTemplateCmd
        com.cloud.utils.exception.CloudRuntimeException: Failed to create templatejava.lang.NullPointerException
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
        at com.cloud.utils.script.Script.execute(Script.java:183)
        at com.cloud.utils.script.Script.execute(Script.java:161)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:363)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:404)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:482)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:532)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:200)
        at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
        at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:416)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:371)
        at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
        at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
        at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
        at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
        at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.CGLIB$createPrivateTemplate$32(<generated>)
        at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095_FastClassByCloudStack_ea3ab7db.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.createPrivateTemplate(<generated>)
        at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)

        at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1369)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
        2013-08-05 17:34:57,413 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Complete async job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ], jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to create templatejava.lang.NullPointerException
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
        at com.cloud.utils.script.Script.execute(Script.java:183)
        at com.cloud.utils.script.Script.execute(Script.java:161)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:363)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:404)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:482)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:532)
        at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:200)
        at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
        at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:416)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:371)
        at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
        at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
        at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
        at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
        at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.CGLIB$createPrivateTemplate$32(<generated>)
        at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095_FastClassByCloudStack_ea3ab7db.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.createPrivateTemplate(<generated>)
        at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)

        2013-08-05 17:34:59,496 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-13:null) SeqA 3-466: Processing Seq 3-466: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"

        {\n \"connections\": []\n}

        ","wait":0}}] }
        2013-08-05 17:34:59,504 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-13:null) SeqA 3-466: Sending Seq 3-466: { Ans: , MgmtId: 7674049379768, via: 3, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
        2013-08-05 17:35:00,243 DEBUG [cloud.api.ApiServlet] (catalina-exec-13:null) ===START=== 10.144.6.19 – GET command=queryAsyncJobResult&jobId=6b671a92-0cde-4a6b-a8c7-33998e1f8926&response=json&sessionkey=kS%2B6Kqk1tmAqe2ZaWxuYGCHGkzY%3D&_=1375704523306
        2013-08-05 17:35:00,256 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-13:null) Async job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ] completed
        2013-08-05 17:35:00,265 DEBUG [cloud.api.ApiServlet] (catalina-exec-13:null) ===END=== 10.144.6.19 – GET command=queryAsyncJobResult&jobId=6b671a92-0cde-4a6b-a8c7-33998e1f8926&response=json&sessionkey=kS%2B6Kqk1tmAqe2ZaWxuYGCHGkzY%3D&_=1375704523306

        Show
        Sailaja Mada added a comment - 2013-08-05 17:34:57,223 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-19:null) submit async job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ], details: AsyncJobVO {id:80, userId: 3, accountId: 3, sessionKey: null, instanceType: Template, instanceId: 208, cmd: org.apache.cloudstack.api.command.user.template.CreateTemplateCmd, cmdOriginator: null, cmdInfo: {"sessionkey":"kS+6Kqk1tmAqe2ZaWxuYGCHGkzY\u003d","cmdEventType":"TEMPLATE.CREATE","ctxUserId":"3","httpmethod":"GET","osTypeId":"142","isPublic":"true","isdynamicallyscalable":"false","response":"json","id":"208","displayText":"tmpl1","snapshotid":"bb36997c-0e86-499b-9013-3454a881b43c","passwordEnabled":"false","name":"tmpl1","_":"1375704520095","ctxAccountId":"3","ctxStartEventId":"317"} , cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, processStatus: 0, resultCode: 0, result: null, initMsid: 7674049379768, completeMsid: null, lastUpdated: null, lastPolled: null, created: null} 2013-08-05 17:34:57,226 DEBUG [cloud.api.ApiServlet] (catalina-exec-19:null) ===END=== 10.144.6.19 – GET command=createTemplate&response=json&sessionkey=kS%2B6Kqk1tmAqe2ZaWxuYGCHGkzY%3D&snapshotid=bb36997c-0e86-499b-9013-3454a881b43c&name=tmpl1&displayText=tmpl1&osTypeId=142&isPublic=true&passwordEnabled=false&isdynamicallyscalable=false&_=1375704520095 2013-08-05 17:34:57,229 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Executing org.apache.cloudstack.api.command.user.template.CreateTemplateCmd for job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ] 2013-08-05 17:34:57,245 DEBUG [cloud.user.AccountManagerImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Access to Acct [3-user1] granted to Acct [3-user1] by DomainChecker_EnhancerByCloudStack_d17a7327 2013-08-05 17:34:57,283 DEBUG [storage.image.TemplateDataFactoryImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) template 208 is already in store:2, type:Image 2013-08-05 17:34:57,297 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) copyAsync inspecting src type SNAPSHOT copyAsync inspecting dest type TEMPLATE 2013-08-05 17:34:57,331 INFO [storage.endpoint.DefaultEndPointSelector] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) No running ssvm is found, so command will be sent to LocalHostEndPoint 2013-08-05 17:34:57,362 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Unable to create mount point: /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476 2013-08-05 17:34:57,363 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Unable to create mount point: /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476 2013-08-05 17:34:57,365 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Executing: null -p /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476/snapshots/3/16 -s 3c4326d9-bf4e-45cf-81d4-7024f8efedd5.vhd -n bdb99a9e-94d7-43e1-9f00-beaa77ad7748.vhd -t /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476/template/tmpl/3/208 2013-08-05 17:34:57,365 WARN [storage.resource.NfsSecondaryStorageResource] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Exception: null -p /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476/snapshots/3/16 -s 3c4326d9-bf4e-45cf-81d4-7024f8efedd5.vhd -n bdb99a9e-94d7-43e1-9f00-beaa77ad7748.vhd -t /var/lib/cloud/mnt/secStorage/67bdb259-cd09-38e0-8672-ac90987f7476/template/tmpl/3/208 java.lang.NullPointerException at java.lang.ProcessBuilder.start(ProcessBuilder.java:457) at com.cloud.utils.script.Script.execute(Script.java:183) at com.cloud.utils.script.Script.execute(Script.java:161) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:363) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:404) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:482) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:532) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:200) at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76) at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93) at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:416) at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:371) at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55) at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549) at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556) at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 2013-08-05 17:34:57,386 DEBUG [cloud.template.TemplateManagerImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Failed to create templatejava.lang.NullPointerException at java.lang.ProcessBuilder.start(ProcessBuilder.java:457) at com.cloud.utils.script.Script.execute(Script.java:183) at com.cloud.utils.script.Script.execute(Script.java:161) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:363) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:404) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:482) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:532) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:200) at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76) at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93) at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:416) at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:371) at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55) at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549) at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556) at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355) at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.CGLIB$createPrivateTemplate$32(<generated>) at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095_FastClassByCloudStack_ea3ab7db.invoke(<generated>) at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.createPrivateTemplate(<generated>) at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 2013-08-05 17:34:57,408 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Unexpected exception while executing org.apache.cloudstack.api.command.user.template.CreateTemplateCmd com.cloud.utils.exception.CloudRuntimeException: Failed to create templatejava.lang.NullPointerException at java.lang.ProcessBuilder.start(ProcessBuilder.java:457) at com.cloud.utils.script.Script.execute(Script.java:183) at com.cloud.utils.script.Script.execute(Script.java:161) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:363) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:404) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:482) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:532) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:200) at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76) at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93) at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:416) at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:371) at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55) at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549) at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556) at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355) at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.CGLIB$createPrivateTemplate$32(<generated>) at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095_FastClassByCloudStack_ea3ab7db.invoke(<generated>) at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.createPrivateTemplate(<generated>) at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1369) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 2013-08-05 17:34:57,413 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-23:job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ]) Complete async job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ], jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to create templatejava.lang.NullPointerException at java.lang.ProcessBuilder.start(ProcessBuilder.java:457) at com.cloud.utils.script.Script.execute(Script.java:183) at com.cloud.utils.script.Script.execute(Script.java:161) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:363) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:404) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:482) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:532) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:200) at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76) at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93) at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:416) at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:371) at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55) at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549) at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556) at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355) at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.CGLIB$createPrivateTemplate$32(<generated>) at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095_FastClassByCloudStack_ea3ab7db.invoke(<generated>) at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_f5953095.createPrivateTemplate(<generated>) at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 2013-08-05 17:34:59,496 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-13:null) SeqA 3-466: Processing Seq 3-466: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":" {\n \"connections\": []\n} ","wait":0}}] } 2013-08-05 17:34:59,504 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-13:null) SeqA 3-466: Sending Seq 3-466: { Ans: , MgmtId: 7674049379768, via: 3, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] } 2013-08-05 17:35:00,243 DEBUG [cloud.api.ApiServlet] (catalina-exec-13:null) ===START=== 10.144.6.19 – GET command=queryAsyncJobResult&jobId=6b671a92-0cde-4a6b-a8c7-33998e1f8926&response=json&sessionkey=kS%2B6Kqk1tmAqe2ZaWxuYGCHGkzY%3D&_=1375704523306 2013-08-05 17:35:00,256 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-13:null) Async job-80 = [ 6b671a92-0cde-4a6b-a8c7-33998e1f8926 ] completed 2013-08-05 17:35:00,265 DEBUG [cloud.api.ApiServlet] (catalina-exec-13:null) ===END=== 10.144.6.19 – GET command=queryAsyncJobResult&jobId=6b671a92-0cde-4a6b-a8c7-33998e1f8926&response=json&sessionkey=kS%2B6Kqk1tmAqe2ZaWxuYGCHGkzY%3D&_=1375704523306
        Sailaja Mada made changes -
        Attachment apilog.log [ 12596102 ]
        Attachment management-server.log [ 12596103 ]
        Attachment postupgradecloud-backup.dmp [ 12596104 ]
        Attachment cloud.log [ 12596105 ]
        Attachment sysvm.log [ 12596106 ]
        Hide
        manasaveloori added a comment -

        Observed the issue on VMware also on upgraded environment.

        Show
        manasaveloori added a comment - Observed the issue on VMware also on upgraded environment.
        Animesh Chaturvedi made changes -
        Component/s Upgrade [ 12320817 ]
        Hide
        ASF subversion and git services added a comment -

        Commit 41f7820709f8856a93473818f13093c6dd6dd7ca in branch refs/heads/4.2 from Devdeep Singh
        [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=41f7820 ]

        CLOUDSTACK-3774: NPE while creating template from snapshot on a upgraded setup.
        After upgrade the SSVM was staying in connecting/alert state. This was because
        while handling the processConnect command for ssvm the management server was
        trying to template sync. For resource limit calculation it was trying to get the
        remote size of the template. If the template was no longer available a number
        format exception was thrown. The process connect wasn't getting completed and
        the ssvm used to stay in connecting/alert state. While creating template from
        snapshot cloudstack looks for up and enabled ssvms and because there wasn't
        any available (as the ssvm was in coonecting/alert state) it used to pick the
        wrong resource (LocalNfs*Resource) instead of the NfsSecondaryStorageResource.

        Fixed the issue by making sure number format exceptions are avoided so that
        SSVM moves to the right state.

        Show
        ASF subversion and git services added a comment - Commit 41f7820709f8856a93473818f13093c6dd6dd7ca in branch refs/heads/4.2 from Devdeep Singh [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=41f7820 ] CLOUDSTACK-3774 : NPE while creating template from snapshot on a upgraded setup. After upgrade the SSVM was staying in connecting/alert state. This was because while handling the processConnect command for ssvm the management server was trying to template sync. For resource limit calculation it was trying to get the remote size of the template. If the template was no longer available a number format exception was thrown. The process connect wasn't getting completed and the ssvm used to stay in connecting/alert state. While creating template from snapshot cloudstack looks for up and enabled ssvms and because there wasn't any available (as the ssvm was in coonecting/alert state) it used to pick the wrong resource (LocalNfs*Resource) instead of the NfsSecondaryStorageResource. Fixed the issue by making sure number format exceptions are avoided so that SSVM moves to the right state.
        Hide
        ASF subversion and git services added a comment -

        Commit 1741fdfb12fd37ce73708b4d563ac22217098c03 in branch refs/heads/master from Devdeep Singh
        [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=1741fdf ]

        CLOUDSTACK-3774: NPE while creating template from snapshot on a upgraded setup.
        After upgrade the SSVM was staying in connecting/alert state. This was because
        while handling the processConnect command for ssvm the management server was
        trying to template sync. For resource limit calculation it was trying to get the
        remote size of the template. If the template was no longer available a number
        format exception was thrown. The process connect wasn't getting completed and
        the ssvm used to stay in connecting/alert state. While creating template from
        snapshot cloudstack looks for up and enabled ssvms and because there wasn't
        any available (as the ssvm was in coonecting/alert state) it used to pick the
        wrong resource (LocalNfs*Resource) instead of the NfsSecondaryStorageResource.

        Fixed the issue by making sure number format exceptions are avoided so that
        SSVM moves to the right state.

        Show
        ASF subversion and git services added a comment - Commit 1741fdfb12fd37ce73708b4d563ac22217098c03 in branch refs/heads/master from Devdeep Singh [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=1741fdf ] CLOUDSTACK-3774 : NPE while creating template from snapshot on a upgraded setup. After upgrade the SSVM was staying in connecting/alert state. This was because while handling the processConnect command for ssvm the management server was trying to template sync. For resource limit calculation it was trying to get the remote size of the template. If the template was no longer available a number format exception was thrown. The process connect wasn't getting completed and the ssvm used to stay in connecting/alert state. While creating template from snapshot cloudstack looks for up and enabled ssvms and because there wasn't any available (as the ssvm was in coonecting/alert state) it used to pick the wrong resource (LocalNfs*Resource) instead of the NfsSecondaryStorageResource. Fixed the issue by making sure number format exceptions are avoided so that SSVM moves to the right state.
        Devdeep Singh made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Animesh Chaturvedi added a comment -

        These blockers and critical issues are resolved but not verified. Reporters of these issues please verify the fixes and help close these issues

        Show
        Animesh Chaturvedi added a comment - These blockers and critical issues are resolved but not verified. Reporters of these issues please verify the fixes and help close these issues
        Hide
        Sailaja Mada added a comment -

        NPE is not observed now. Hence closing the bug.

        Show
        Sailaja Mada added a comment - NPE is not observed now. Hence closing the bug.
        Sailaja Mada made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        3d 2h 44m 1 Devdeep Singh 27/Jul/13 12:19
        Resolved Resolved Reopened Reopened
        9d 50m 1 Sailaja Mada 05/Aug/13 13:09
        Reopened Reopened Resolved Resolved
        23h 17m 1 Devdeep Singh 06/Aug/13 12:27
        Resolved Resolved Closed Closed
        13d 5h 15m 1 Sailaja Mada 19/Aug/13 17:42

          People

          • Assignee:
            Devdeep Singh
            Reporter:
            Sailaja Mada
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development