Uploaded image for project: 'CloudStack'
  1. CloudStack
  2. CLOUDSTACK-7857

CitrixResourceBase wrongly calculates total memory on hosts with a lot of memory and large Dom0



    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 4.3.0, 4.4.0, 4.5.0, 4.3.1, 4.4.1, 4.6.0
    • None
    • None
    • Security Level: Public (Anyone can view this level - this is the default.)
    • None


      We have hosts with 256GB memory and 4GB dom0. During startup ACS calculates available memory using this formula:
      protected void fillHostInfo
      ram = (long) ((ram - dom0Ram - _xs_memory_used) * _xs_virtualization_factor);
      In our situation:
      ram = 274841497600
      dom0Ram = 4269801472
      _xs_memory_used = 128 * 1024 * 1024L = 134217728
      _xs_virtualization_factor = 63.0/64.0 = 0,984375
      (274841497600 - 4269801472 - 134217728) * 0,984375 = 266211892800

      This is in fact not the actual amount of memory available for instances. The difference in our situation is a little less then 1GB. On this particular hypervisor Dom0+Xen uses about 9GB.
      As the comment above the definition of XsMemoryUsed allready stated it's time to review this logic.
      "//Hypervisor specific params with generic value, may need to be overridden for specific versions"

      The effect of this bug is that when you put a hypervisor in maintenance it might try to move instances (usually small instances (<1GB)) to a host that in fact does not have enought free memory.
      This exception is thrown:

      ERROR [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-3:ctx-09aca6e9 work-8981) Terminating HAWork[8981-Migration-4482-Running-Migrating]
      com.cloud.utils.exception.CloudRuntimeException: Unable to migrate due to Catch Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntim
      eException: Unable to migrate VM(r-4482-VM) from host(6805d06c-4d5b-4438-a245-7915e93041d9) due to Task failed! Task record: uuid: 645b63c8-1426-b412-7b6a-13d61ee7ab2e
      nameLabel: Async.VM.pool_migrate
      allowedOperations: []
      currentOperations: {}
      created: Thu Nov 06 13:44:14 CET 2014
      finished: Thu Nov 06 13:44:14 CET 2014
      status: failure
      residentOn: com.xensource.xenapi.Host@b42882c6
      progress: 1.0
      type: <none/>
      errorInfo: [HOST_NOT_ENOUGH_FREE_MEMORY, 272629760, 263131136]
      otherConfig: {}
      subtaskOf: com.xensource.xenapi.Task@aaf13f6f
      subtasks: []

      at com.cloud.vm.VirtualMachineManagerImpl.migrate(VirtualMachineManagerImpl.java:1840)
      at com.cloud.vm.VirtualMachineManagerImpl.migrateAway(VirtualMachineManagerImpl.java:2214)
      at com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:610)
      at com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.runWithContext(HighAvailabilityManagerImpl.java:865)
      at com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.access$000(HighAvailabilityManagerImpl.java:822)
      at com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread$1.run(HighAvailabilityManagerImpl.java:834)
      at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
      at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
      at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
      at com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:831)




            Unassigned Unassigned
            jvanlieshout@schubergphilis.com Joris van Lieshout
            1 Vote for this issue
            6 Start watching this issue