Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.17
    • Fix Version/s: 0.18
    • Component/s: C++ Clustering
    • Labels:
      None

      Description

      Sporadic failure in ha_tests. 2 types of failure observed:

      ==== failure 1
      ha_tests.LongTests.test_failover_send_receive ....................................... start
      Traceback (most recent call last):
      File "/home/aconway/qpidha/qpid/cpp/src/tests/ha_tests.py", line 884, in test_failover_send_receive
      brokers[(i+1)%3].wait_status("ready")
      File "/home/aconway/qpidha/qpid/cpp/src/tests/ha_tests.py", line 117, in wait_status
      assert retry(try_get_status, timeout=20), "%s status != %r"%(self, status)
      AssertionError: Broker<003:cluster0-0.log qpidd-23 :58163> status != 'ready'
      ha_tests.LongTests.test_failover_send_receive ....................................... fail
      Error during test: Traceback (most recent call last):
      File "/home/aconway/qpidha/opt/src/tests/python/commands/qpid-python-test", line 340, in run
      phase()
      File "/home/aconway/qpidha/qpid/cpp/src/tests/ha_tests.py", line 884, in test_failover_send_receive
      brokers[(i+1)%3].wait_status("ready")
      File "/home/aconway/qpidha/qpid/cpp/src/tests/ha_tests.py", line 117, in wait_status
      assert retry(try_get_status, timeout=20), "%s status != %r"%(self, status)
      AssertionError: Broker<003:cluster0-0.log qpidd-23 :58163> status != 'ready'

      ==== failure 2

      ha_tests.LongTests.test_failover_send_receive ........................................ start
      Traceback (most recent call last):
      File "/home/aconway/qpidha/qpid/cpp/src/tests/ha_tests.py", line 884, in test_failover_send_receive
      brokers[(i+1)%3].wait_status("ready")
      File "/home/aconway/qpidha/qpid/cpp/src/tests/ha_tests.py", line 117, in wait_status
      assert retry(try_get_status, timeout=20), "%s status != %r"%(self, status)
      AssertionError: Broker<003:cluster0-0.log qpidd-23 :47682> status != 'ready'
      ha_tests.LongTests.test_failover_send_receive ........................................ fail
      Error during test: Traceback (most recent call last):
      File "/home/aconway/qpidha/opt/src/tests/python/commands/qpid-python-test", line 340, in run
      phase()
      File "/home/aconway/qpidha/qpid/cpp/src/tests/ha_tests.py", line 884, in test_failover_send_receive
      brokers[(i+1)%3].wait_status("ready")
      File "/home/aconway/qpidha/qpid/cpp/src/tests/ha_tests.py", line 117, in wait_status
      assert retry(try_get_status, timeout=20), "%s status != %r"%(self, status)
      AssertionError: Broker<003:cluster0-0.log qpidd-23 :47682> status != 'ready'

      ==== failure 3

      ha_tests.ReplicationTests.test_send_receive ......................................... fail
      Error during test: Traceback (most recent call last):
      File "/home/aconway/qpidha/opt/src/tests/python/commands/qpid-python-test", line 340, in run
      phase()
      File "/home/aconway/qpidha/qpid/cpp/src/tests/ha_tests.py", line 382, in test_send_receive
      brokers[2].assert_browse_backup("q", expect, transform=sn)
      File "/home/aconway/qpidha/qpid/cpp/src/tests/ha_tests.py", line 158, in assert_browse_backup
      assert_browse_retry(bs, queue, expected, **kwargs)
      File "/home/aconway/qpidha/qpid/cpp/src/tests/brokertest.py", line 466, in assert_browse_retry
      assert expect_contents == actual_contents, msg
      AssertionError: browse failed: [991L, 992L, 993L, 994L, 995L, 996L, 997L, 998L, 999L, 1000L] != [990, 991, 992, 993, 994, 995, 996, 997, 998, 999, 1000]

        Activity

        Hide
        Alan Conway added a comment -

        On 0.18 branch:

        ------------------------------------------------------------------------
        r1363485 | aconway | 2012-07-19 15:16:03 -0400 (Thu, 19 Jul 2012) | 7 lines

        QPID-4138: HA sporadic failure in ha_tests

        This was another test issue: trace logging had been left on by mistake, which was
        slowing down the tests to the point they were hitting test time-outs.

        Also in this patch: Include symbolic satus in membership update log message.

        Show
        Alan Conway added a comment - On 0.18 branch: ------------------------------------------------------------------------ r1363485 | aconway | 2012-07-19 15:16:03 -0400 (Thu, 19 Jul 2012) | 7 lines QPID-4138 : HA sporadic failure in ha_tests This was another test issue: trace logging had been left on by mistake, which was slowing down the tests to the point they were hitting test time-outs. Also in this patch: Include symbolic satus in membership update log message.
        Hide
        Justin Ross added a comment -

        Reviewed by Gordon. Approved for 0.18.

        Show
        Justin Ross added a comment - Reviewed by Gordon. Approved for 0.18.
        Hide
        Gordon Sim added a comment -

        No issue merging this to 0.18, its just log clean up for the tests.

        Show
        Gordon Sim added a comment - No issue merging this to 0.18, its just log clean up for the tests.
        Hide
        Alan Conway added a comment -

        Fixed on trunk, request merge to 0.18

        ------------------------------------------------------------------------
        r1361393 | aconway | 2012-07-13 16:48:05 -0400 (Fri, 13 Jul 2012) | 7 lines

        QPID-4138: HA sporadic failure in ha_tests

        This was another test issue: trace logging had been left on by mistake, which was
        slowing down the tests to the point they were hitting test time-outs.

        Also in this patch: Include symbolic satus in membership update log message.

        ------------------------------------------------------------------------

        Show
        Alan Conway added a comment - Fixed on trunk, request merge to 0.18 ------------------------------------------------------------------------ r1361393 | aconway | 2012-07-13 16:48:05 -0400 (Fri, 13 Jul 2012) | 7 lines QPID-4138 : HA sporadic failure in ha_tests This was another test issue: trace logging had been left on by mistake, which was slowing down the tests to the point they were hitting test time-outs. Also in this patch: Include symbolic satus in membership update log message. ------------------------------------------------------------------------

          People

          • Assignee:
            Alan Conway
            Reporter:
            Alan Conway
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development