Uploaded image for project: 'CloudStack'
  1. CloudStack
  2. CLOUDSTACK-3542

Ceph RBD provides wrong usage info to CloudStack

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.1.0, 4.2.0
    • Fix Version/s: None
    • Component/s: Doc
    • Security Level: Public (Anyone can view this level - this is the default.)
    • Labels:
    • Environment:
      Ubuntu 12.04.2

      Description

      I am installing CloudStack 4.1.0 to use Ceph RBD as primary storage. I failed to create VM instance using the RBD as primary storage because Ceph provides wrong disk usage information. See below:

      root@hv-kvm-02:~# virsh pool-info bab81ce8-d53f-3a7d-b8f6-841702f65c89
      Name: bab81ce8-d53f-3a7d-b8f6-841702f65c89
      UUID: bab81ce8-d53f-3a7d-b8f6-841702f65c89
      State: running
      Persistent: no
      Autostart: no
      Capacity: 5.47 TiB
      Allocation: 34819.02 TiB <-- wrong information
      Available: 5.47 TiB

      Record on the "storage_pool" table on "cloud" database:

      ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

      id name uuid pool_type port data_center_id pod_id cluster_id available_bytes capacity_bytes host_address user_info path created removed update_time status storage_provider_id scope

      ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

      209 sc-image bab81ce8-d53f-3a7d-b8f6-841702f65c89 RBD 6789 6 6 6 38283921137336466 6013522722816 ceph-mon.xxx.com admin:xxx sc1 2013-07-13 08:58:27 NULL NULL Up NULL NULL

      ===
      2013-07-15 11:27:31,632 DEBUG [cloud.alert.AlertManagerImpl] (CapacityChecker:null) System Alert: Low Available Storage in cluster Cluster-01 pod Pod-01 of availability zone 01
      2013-07-15 11:27:31,632 DEBUG [cloud.alert.AlertManagerImpl] (CapacityChecker:null) Available storage space is low, total: 13282342 MB, used: 36510398411 MB (274879.22%)
      ===

      As a result, VM instance creation fails since it's not able to find available storage pool.

      ===
      2013-07-15 11:15:28,313 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-3:job-168) Checking pool: 208 for volume allocation [Vol[227|vm=225|ROOT]], maxSize : 15828044742656, totalAllocatedSize : 1769538048, askingSize : 8589934592, allocated disable threshold: 0.85
      2013-07-15 11:15:28,313 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Checking if storage pool is suitable, name: sc-image ,poolId: 209
      2013-07-15 11:15:28,313 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Is localStorageAllocationNeeded? false
      2013-07-15 11:15:28,313 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Is storage pool shared? true
      2013-07-15 11:15:28,317 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-3:job-168) Checking pool 209 for storage, totalSize: 6013522722816, usedBytes: 38283921137336466, usedPct: 6366.305226067051, disable threshold: 0.85
      2013-07-15 11:15:28,317 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-3:job-168) Insufficient space on pool: 209 since its usage percentage: 6366.305226067051 has crossed the pool.storage.capacity.disablethreshold: 0.85
      2013-07-15 11:15:28,317 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-3:job-168) FirstFitStoragePoolAllocator returning 1 suitable storage pools
      2013-07-15 11:15:28,317 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) Checking suitable pools for volume (Id, Type): (228,DATADISK)
      2013-07-15 11:15:28,317 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) We need to allocate new storagepool for this volume
      2013-07-15 11:15:28,319 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) Calling StoragePoolAllocators to find suitable pools
      2013-07-15 11:15:28,319 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-3:job-168) Looking for pools in dc: 6 pod:6 cluster:6 having tags:[rbd]
      2013-07-15 11:15:28,322 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-3:job-168) FirstFitStoragePoolAllocator has 1 pools to check for allocation
      2013-07-15 11:15:28,322 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Checking if storage pool is suitable, name: sc-image ,poolId: 209
      2013-07-15 11:15:28,322 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Is localStorageAllocationNeeded? false
      2013-07-15 11:15:28,322 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Is storage pool shared? true
      2013-07-15 11:15:28,326 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-3:job-168) Checking pool 209 for storage, totalSize: 6013522722816, usedBytes: 38283921137336466, usedPct: 6366.305226067051, disable threshold: 0.85
      2013-07-15 11:15:28,326 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-3:job-168) Insufficient space on pool: 209 since its usage percentage: 6366.305226067051 has crossed the pool.storage.capacity.disablethreshold: 0.85
      2013-07-15 11:15:28,326 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-3:job-168) FirstFitStoragePoolAllocator returning 0 suitable storage pools
      2013-07-15 11:15:28,326 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) No suitable pools found for volume: Vol[228|vm=225|DATADISK] under cluster: 6
      2013-07-15 11:15:28,326 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) No suitable pools found
      2013-07-15 11:15:28,326 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) No suitable storagePools found under this Cluster: 6
      2013-07-15 11:15:28,326 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) Could not find suitable Deployment Destination for this VM under any clusters, returning.
      2013-07-15 11:15:28,332 DEBUG [cloud.vm.UserVmManagerImpl] (Job-Executor-3:job-168) Destroying vm VM[User|Indra-Test-3] as it failed to create on Host with Id:null
      2013-07-15 11:15:28,498 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-3:job-168) VM state transitted from :Stopped to Error with event: OperationFailedToErrorvm's original host id: null new host id: null host id before state transition: null
      2013-07-15 11:15:29,125 INFO [user.vm.DeployVMCmd] (Job-Executor-3:job-168) com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|Indra-Test-3]Scope=interface com.cloud.dc.DataCenter; id=6
      ===

        Attachments

          Activity

            People

            • Assignee:
              sadhu sadhu suresh
              Reporter:
              indrapr Indra Pramana

              Dates

              • Created:
                Updated:

                Issue deployment