Uploaded image for project: 'CloudStack'
  1. CloudStack
  2. CLOUDSTACK-9989

Extend smoketests suite

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: marvin
    • Security Level: Public (Anyone can view this level - this is the default.)
    • Labels:
      None

      Description

      This improvement is to identify areas in smoketests to improve.

      Item we could move from components tests into smoketests:
      test_accounts.py
      test_host_maintenance.py
      test_regions.py
      test_usage.py
      test_projects

      • test_usage.py:
        [root@trl-722-k-cs410-bstoyanov-marvin ~]# cat /marvin//MarvinLogs/test_usage_QZE99I/failed_plus_exceptions.txt
        2017-07-07 08:50:54,314 - CRITICAL - EXCEPTION: None: ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/site-packages/nose/suite.py", line 209, in run\n    self.setUp()\n', '  File "/usr/lib/python2.7/site-packages/nose/suite.py", line 292, in setUp\n    self.setupContext(ancestor)\n', '  File "/usr/lib/python2.7/site-packages/nose/suite.py", line 315, in setupContext\n    try_run(context, names)\n', '  File "/usr/lib/python2.7/site-packages/nose/util.py", line 471, in try_run\n    return func()\n', '  File "/marvin/tests/component/test_usage.py", line 1023, in setUpClass\n    cls.iso.id\n', 'Exception: ISO download failed exception: Failed to download ISO: ad2e9212-5aef-461e-9f7d-7cc86044f1ce\n']
        2017-07-07 09:09:18,458 - CRITICAL - EXCEPTION: test_01_template_usage: ['Traceback (most recent call last):\n', '  File "/usr/lib64/python2.7/unittest/case.py", line 369, in run\n    testMethod()\n', '  File "/marvin/tests/component/test_usage.py", line 928, in test_01_template_usage\n    self.template.delete(self.userapiclient)\n', '  File "/usr/lib/python2.7/site-packages/marvin/lib/base.py", line 1333, in delete\n    apiclient.deleteTemplate(cmd)\n', '  File "/usr/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 1874, in deleteTemplate\n    response = self.connection.marvinRequest(command, response_type=response, method=method)\n', '  File "/usr/lib/python2.7/site-packages/marvin/cloudstackConnection.py", line 379, in marvinRequest\n    raise e\n', "Exception: Job failed: {jobprocstatus : 0, created : u'2017-07-07T09:09:05+0000', jobresult : {errorcode : 530, errortext : u'Failed to delete template'}, cmd : u'org.apache.cloudstack.api.command.user.template.DeleteTemplateCmd', userid : u'ad0cc9a7-2b64-404d-8b93-7fb64642e6a5', jobstatus : 2, jobid : u'5e1a9a95-164d-40b0-af84-1bcd6713c513', jobresultcode : 530, jobinstanceid : u'47998d73-3ee5-4535-9c43-82f2b47d2d5d', jobresulttype : u'object', jobinstancetype : u'Template', accountid : u'e3104721-4a36-42ac-9238-1e7d0583f9ff'}\n"]
        2017-07-07 09:16:08,697 - CRITICAL - EXCEPTION: test_01_volume_usage: ['Traceback (most recent call last):\n', '  File "/usr/lib64/python2.7/unittest/case.py", line 369, in run\n    testMethod()\n', '  File "/marvin/tests/component/test_usage.py", line 693, in test_01_volume_usage\n    "url"] = self.testdata["coreos_volume"]["urlkvm"]\n', "KeyError: 'coreos_volume'\n"]
        
        Ran 8 tests in 3271.872s
        
        FAILED (errors=3)
        

      Test regions:

      Test for duplicate checks on region id ... === TestName: test_createRegionWithExistingRegionId | Status : SUCCESS ===
      ok
      Test for duplicate checks on region name ... === TestName: test_createRegionWithExistingRegionName | Status : SUCCESS ===
      ok
      Test for update Region ... === TestName: test_updateRegion | Status : SUCCESS ===
      ok
      
      ----------------------------------------------------------------------
      Ran 3 tests in 0.250s
      
      OK

      test_host_maintance.py:
      It's being skipped if hosts are <2, can be refactored to <= 2.

      test_01_cancel_host_maintenace_with_no_migration_jobs (tests.component.test_host_maintenance.TestHostMaintenance) ... SKIP: Cancel host maintenance when VMs are migrating should be tested for 2 or more hosts
      test_02_cancel_host_maintenace_with_migration_jobs (tests.component.test_host_maintenance.TestHostMaintenance) ... SKIP: Cancel host maintenance when VMs are migrating can only be tested with 2 hosts
      
      ----------------------------------------------------------------------
      Ran 2 tests in 0.118s
      
      OK (SKIP=2)

      test_accounts.py

      ----------------------------------------------------------------------
      
      [root@trl-722-k-cs410-bstoyanov-marvin ~]# cat /marvin//MarvinLogs/test_accounts_9JJBCO/failed_plus_exceptions.txt
      2017-07-07 10:41:13,365 - CRITICAL - EXCEPTION: test_01_service_offering_siblings: ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/site-packages/nose/suite.py", line 209, in run\n    self.setUp()\n', '  File "/usr/lib/python2.7/site-packages/nose/suite.py", line 292, in setUp\n    self.setupContext(ancestor)\n', '  File "/usr/lib/python2.7/site-packages/nose/suite.py", line 315, in setupContext\n    try_run(context, names)\n', '  File "/usr/lib/python2.7/site-packages/nose/util.py", line 471, in try_run\n    return func()\n', '  File "/marvin/tests/component/test_accounts.py", line 827, in setUpClass\n    builtin_info = get_builtin_template_info(cls.api_client, cls.zone.id)\n', '  File "/usr/lib/python2.7/site-packages/marvin/lib/common.py", line 502, in get_builtin_template_info\n    zoneid)\n', '  File "/usr/lib/python2.7/site-packages/marvin/lib/base.py", line 1289, in extract\n    return apiclient.extractTemplate(cmd)\n', '  File "/usr/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 2939, in extractTemplate\n    response = self.connection.marvinRequest(command, response_type=response, method=method)\n', '  File "/usr/lib/python2.7/site-packages/marvin/cloudstackConnection.py", line 379, in marvinRequest\n    raise e\n', "Exception: Job failed: {jobprocstatus : 0, created : u'2017-07-07T10:41:00+0000', jobresult : {errorcode : 530, errortext : u'Unable to create a link for entity at template/tmpl/1/4/a647a939-5764-3211-8f93-0c7357de21cf.qcow2 on ssvm,No remote endpoint to send command, check if host or ssvm is down?'}, cmd : u'org.apache.cloudstack.api.command.user.template.ExtractTemplateCmd', userid : u'fe25b2c4-5b42-11e7-bd75-0635a0010739', jobstatus : 2, jobid : u'18248038-e3d3-41c6-99dd-982fed3c13c2', jobresultcode : 530, jobinstanceid : u'c7373f24-5b42-11e7-bd75-0635a0010739', jobresulttype : u'object', jobinstancetype : u'Template', accountid : u'fe25a83f-5b42-11e7-bd75-0635a0010739'}\n"]
      2017-07-07 10:41:15,811 - CRITICAL - FAILED: test_user_key_renew_same_account: ['Traceback (most recent call last):\n', '  File "/usr/lib64/python2.7/unittest/case.py", line 369, in run\n    testMethod()\n', '  File "/marvin/tests/component/test_accounts.py", line 1655, in test_user_key_renew_same_account\n    "Check User having secret key")\n', '  File "/usr/lib64/python2.7/unittest/case.py", line 553, in assertEqual\n    assertion_func(first, second, msg=msg)\n', '  File "/usr/lib64/python2.7/unittest/case.py", line 546, in _baseAssertEqual\n    raise self.failureException(msg)\n', 'AssertionError: Check User having secret key\n']
      2017-07-07 10:41:54,559 - CRITICAL - EXCEPTION: test_LoginApiDomain: ['Traceback (most recent call last):\n', '  File "/usr/lib64/python2.7/unittest/case.py", line 369, in run\n    testMethod()\n', '  File "/marvin/tests/component/test_accounts.py", line 1522, in test_LoginApiDomain\n    domainid=domain.id)\n', '  File "/usr/lib/python2.7/site-packages/marvin/lib/base.py", line 315, in login\n    return apiclient.login(cmd)\n', '  File "/usr/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 1639, in login\n    response = self.connection.marvinRequest(command, response_type=response, method=method)\n', '  File "/usr/lib/python2.7/site-packages/marvin/cloudstackConnection.py", line 379, in marvinRequest\n    raise e\n', 'ValueError: No JSON object could be decoded\n']
      2017-07-07 10:42:00,288 - CRITICAL - EXCEPTION: test_LoginApiUuidResponse: ['Traceback (most recent call last):\n', '  File "/usr/lib64/python2.7/unittest/case.py", line 369, in run\n    testMethod()\n', '  File "/marvin/tests/component/test_accounts.py", line 1450, in test_LoginApiUuidResponse\n    password=self.services["account"]["password"]\n', '  File "/usr/lib/python2.7/site-packages/marvin/lib/base.py", line 315, in login\n    return apiclient.login(cmd)\n', '  File "/usr/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 1639, in login\n    response = self.connection.marvinRequest(command, response_type=response, method=method)\n', '  File "/usr/lib/python2.7/site-packages/marvin/cloudstackConnection.py", line 379, in marvinRequest\n    raise e\n', 'ValueError: No JSON object could be decoded\n']
      Ran 15 tests in 1222.303s
      
      FAILED (errors=3, failures=1)
      
      

      test_projects looking good:

      [root@trl-722-k-cs410-bstoyanov-marvin ~]# cat /marvin//MarvinLogs/test_projects_5JAJZW/results.txt
      Verify No cross domain projects ... === TestName: test_02_cross_domain_account_add | Status : SUCCESS ===
      ok
      Test As long as the project exists, its owner can't be removed ... === TestName: test_03_delete_account_with_project | Status : SUCCESS ===
      ok
      Test Verify delete domain with cleanup=true should delete projects ... === TestName: test_04_delete_domain_with_project | Status : SUCCESS ===
      ok
      Verify an account can own multiple projects and can belong to multiple projects ... === TestName: test_01_create_multiple_projects_by_account | Status : SUCCESS ===
      ok
      Test Verify a project user can be later promoted to become a ... === TestName: test_05_user_project_owner_promotion | Status : SUCCESS ===
      ok
      Test Verify there can only be one owner of a project at a time ... === TestName: test_06_max_one_project_owner | Status : SUCCESS ===
      ok
      Test Verify after an account is removed from the project, all its resources stay with the project. ... === TestName: test_07_project_resources_account_delete | Status : SUCCESS ===
      ok
      Test accounts are unassigned from project after project deletion ... === TestName: test_08_cleanup_after_project_delete | Status : SUCCESS ===
      ok
      Test Verify after an account is removed from the project, all his ... === TestName: test_09_project_suspend | Status : SUCCESS ===
      ok
      Test project activation after suspension ... === TestName: test_10_project_activation | Status : SUCCESS ===
      ok
      
      ----------------------------------------------------------------------
      Ran 10 tests in 699.622s
      
      OK

      all these areas are not covered in the smoketests

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                bstoyanov Boris Stoyanov
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: