Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
4.10.0.0
-
Security Level: Public (Anyone can view this level - this is the default.)
-
None
Description
Steps:
1. Install and configure a zone (advanced or basic).
2. Set config storage.cleanup.enabled = true and storage.cleanup.interval = 10 seconds
3. Deploy 1000 VMs and then destroy over multiple iterations.
NPE seen in MS logs while deleting volume:
2015-06-18 16:27:47,797 DEBUG [c.c.v.VirtualMachineManagerImpl] (UserVm-Scavenger-1:ctx-5ecc886e) (logid:132e3ff8) Cleaning up hypervisor data structures (ex. SRs in XenServer) for managed storage
2015-06-18 16:27:47,799 DEBUG [o.a.c.e.o.VolumeOrchestrator] (UserVm-Scavenger-1:ctx-5ecc886e) (logid:132e3ff8) Cleaning storage for vm: 2894
2015-06-18 16:27:47,823 INFO [o.a.c.s.v.VolumeServiceImpl] (UserVm-Scavenger-1:ctx-5ecc886e) (logid:132e3ff8) Expunge volume with no data store specified
2015-06-18 16:27:47,828 DEBUG [c.c.s.StorageManagerImpl] (StorageManager-Scavenger-1:ctx-5e7b4eda) (logid:bb642325) Storage pool garbage collector found 0 templates to clean up in storage pool: XenRT-Zone-0-Pod-0-Cluster-0-Primary-Store-0
2015-06-18 16:27:47,828 INFO [o.a.c.s.v.VolumeServiceImpl] (UserVm-Scavenger-1:ctx-5ecc886e) (logid:132e3ff8) Volume 2894 is not referred anywhere, remove it from volumes table
2015-06-18 16:27:47,829 DEBUG [c.c.s.StorageManagerImpl] (StorageManager-Scavenger-1:ctx-5e7b4eda) (logid:bb642325) Storage pool garbage collector found 0 templates to clean up in storage pool: XenRT-Zone-0-Pod-0-Cluster-1-Primary-Store-0
2015-06-18 16:27:47,832 DEBUG [c.c.s.StorageManagerImpl] (StorageManager-Scavenger-1:ctx-5e7b4eda) (logid:bb642325) Secondary storage garbage collector found 0 templates to cleanup on template_store_ref for store: nfs://10.81.56.7/xenrtnfs/1092931-dycPsK
2015-06-18 16:27:47,833 DEBUG [c.c.s.StorageManagerImpl] (StorageManager-Scavenger-1:ctx-5e7b4eda) (logid:bb642325) Secondary storage garbage collector found 0 snapshots to cleanup on snapshot_store_ref for store: nfs://10.81.56.7/xenrtnfs/1092931-dycPsK
2015-06-18 16:27:47,834 DEBUG [c.c.s.StorageManagerImpl] (StorageManager-Scavenger-1:ctx-5e7b4eda) (logid:bb642325) Secondary storage garbage collector found 0 volumes to cleanup on volume_store_ref for store: nfs://10.81.56.7/xenrtnfs/1092931-dycPsK
2015-06-18 16:27:47,842 DEBUG [c.c.v.VirtualMachineManagerImpl] (UserVm-Scavenger-1:ctx-5ecc886e) (logid:132e3ff8) Expunged VM[User|i-10-2894-VM]
2015-06-18 16:27:47,844 WARN [c.c.s.StorageManagerImpl] (StorageManager-Scavenger-1:ctx-5e7b4eda) (logid:bb642325) Unable to destroy volume 0b22f54b-3242-49ef-b16d-1c7801d5c2bd
java.lang.NullPointerException
at org.apache.cloudstack.storage.volume.VolumeServiceImpl.expungeVolumeAsync(VolumeServiceImpl.java:276)
at com.cloud.storage.StorageManagerImpl.cleanupStorage(StorageManagerImpl.java:1121)
at com.cloud.storage.StorageManagerImpl$StorageGarbageCollector.runInContext(StorageManagerImpl.java:1481)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2015-06-18 16:27:47,850 DEBUG [c.c.u.AccountManagerImpl] (UserVm-Scavenger-1:ctx-5ecc886e) (logid:132e3ff8) Access granted to Acct[ed48b7f2-15a0-11e5-96dd-d275a7df156a-system] to Domain:1/ by AffinityGroupAccessChecker
2015-06-18 16:27:47,871 DEBUG [c.c.v.UserVmManagerImpl] (UserVm-Scavenger-1:ctx-5ecc886e) (logid:132e3ff8) Starting cleaning up vm VM[User|i-10-2894-VM] resources...
Attachments
Issue Links
- links to
