Uploaded image for project: 'Qpid Dispatch'
  1. Qpid Dispatch
  2. DISPATCH-2323

[http1] system_tests_http1_adaptor failure due to Address already in use issue (second round)

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Won't Fix
    • 1.15.0
    • None
    • Protocol Adaptors
    • None

    Description

      https://github.com/apache/qpid-dispatch/runs/4993651356?check_suite_focus=true#step:27:1904

      36/37 Test #70: system_tests_http1_adaptor ........................***Failed  351.39 sec
      ============================= test session starts ==============================
      platform linux -- Python 3.6.8, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /usr/bin/python3
      cachedir: .pytest_cache
      rootdir: /__w/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/tests, configfile: tox.ini
      collecting ... collected 28 items
      
      ::Http1AdaptorManagementTest::test_01_create_delete FAILED
      ::Http1AdaptorManagementTest::test_01_delete_active_connector FAILED
      ::Http1AdaptorOneRouterTest::test_000_stats PASSED
      ::Http1AdaptorOneRouterTest::test_001_get PASSED
      ::Http1AdaptorOneRouterTest::test_002_head PASSED
      ::Http1AdaptorOneRouterTest::test_003_post PASSED
      ::Http1AdaptorOneRouterTest::test_004_put PASSED
      ::Http1AdaptorOneRouterTest::test_005_get_10 PASSED
      ::Http1AdaptorOneRouterTest::test_006_head_10 PASSED
      ::Http1AdaptorOneRouterTest::test_007_post_10 PASSED
      ::Http1AdaptorOneRouterTest::test_008_put_10 PASSED
      ::Http1AdaptorEdge2EdgeTest::test_01_concurrent_requests ERROR
      ::Http1AdaptorEdge2EdgeTest::test_02_credit_replenish ERROR
      ::Http1AdaptorEdge2EdgeTest::test_03_server_reconnect ERROR
      ::Http1AdaptorEdge2EdgeTest::test_04_server_pining_for_the_fjords ERROR
      ::Http1AdaptorEdge2EdgeTest::test_05_large_streaming_msg ERROR
      ::Http1AdaptorEdge2EdgeTest::test_1001_client_request_close ERROR
      ::Http1AdaptorEdge2EdgeTest::test_1002_client_response_close ERROR
      ::Http1AdaptorEdge2EdgeTest::test_2000_curl_get ERROR
      ::Http1AdaptorEdge2EdgeTest::test_2001_curl_put ERROR
      ::Http1AdaptorEdge2EdgeTest::test_2002_curl_post ERROR
      ::Http1AdaptorBadEndpointsTest::test_01_unsolicited_response PASSED
      ::Http1AdaptorBadEndpointsTest::test_02_bad_request_message PASSED
      ::Http1AdaptorBadEndpointsTest::test_03_bad_response_message PASSED
      ::Http1AdaptorBadEndpointsTest::test_04_client_request_close PASSED
      ::Http1AdaptorBadEndpointsTest::test_05_client_response_close PASSED
      ::Http1AdaptorQ2Standalone::test_01_backpressure_client PASSED
      ::Http1AdaptorQ2Standalone::test_02_backpressure_server PASSED
      
      ==================================== ERRORS ====================================
      ___ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_01_concurrent_requests ____
      
      cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'>
      name = 'EA2', mode = 'edge'
      extra = [('connector', {'host': '127.0.0.1', 'idleTimeoutSeconds': '120', 'name': 'uplink', 'port': 25519, ...}), ('httpConnec...protocolVersion': 'HTTP1'}), ('httpConnector', {'address': 'testServer10', 'port': 25522, 'protocolVersion': 'HTTP1'})]
      
          @classmethod
          def router(cls, name, mode, extra):
              config = [
                  ('router', {'mode': mode,
                              'id': name,
                              'allowUnsettledMulticast': 'yes'}),
                  ('listener', {'role': 'normal',
                                'port': cls.tester.get_port()}),
                  ('address', {'prefix': 'closest', 'distribution': 'closest'}),
                  ('address', {'prefix': 'multicast', 'distribution': 'multicast'}),
              ]
          
              if extra:
                  config.extend(extra)
              config = Qdrouterd.Config(config)
      >       cls.routers.append(cls.tester.qdrouterd(name, config, wait=True))
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: in qdrouterd
          return self.cleanup(Qdrouterd(*args, **kwargs))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: in __init__
          self.wait_ready()
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: in wait_ready
          self.wait_connectors(**retry_kwargs)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {}
      @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>
      @py_assert4 = None
      @py_format6 = 'Port not connected 25519\n>assert None\n{None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})\n}'
      
          def wait_connectors(self, **retry_kwargs):
              """
              Wait for all connectors to be connected
              @param retry_kwargs: keyword args for L{retry}
              """
              for c in self.config.sections('connector'):
      >           assert retry(lambda: self.is_connected(port=c['port'], host=self.get_host(c.get('protocolFamily'))),
                               **retry_kwargs), "Port not connected %s" % c['port']
      E           AssertionError: Port not connected 25519
      E           assert None
      E            +  where None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: AssertionError
      _____ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_02_credit_replenish _____
      
      cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'>
      name = 'EA2', mode = 'edge'
      extra = [('connector', {'host': '127.0.0.1', 'idleTimeoutSeconds': '120', 'name': 'uplink', 'port': 25519, ...}), ('httpConnec...protocolVersion': 'HTTP1'}), ('httpConnector', {'address': 'testServer10', 'port': 25522, 'protocolVersion': 'HTTP1'})]
      
          @classmethod
          def router(cls, name, mode, extra):
              config = [
                  ('router', {'mode': mode,
                              'id': name,
                              'allowUnsettledMulticast': 'yes'}),
                  ('listener', {'role': 'normal',
                                'port': cls.tester.get_port()}),
                  ('address', {'prefix': 'closest', 'distribution': 'closest'}),
                  ('address', {'prefix': 'multicast', 'distribution': 'multicast'}),
              ]
          
              if extra:
                  config.extend(extra)
              config = Qdrouterd.Config(config)
      >       cls.routers.append(cls.tester.qdrouterd(name, config, wait=True))
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: in qdrouterd
          return self.cleanup(Qdrouterd(*args, **kwargs))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: in __init__
          self.wait_ready()
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: in wait_ready
          self.wait_connectors(**retry_kwargs)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {}
      @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>
      @py_assert4 = None
      @py_format6 = 'Port not connected 25519\n>assert None\n{None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})\n}'
      
          def wait_connectors(self, **retry_kwargs):
              """
              Wait for all connectors to be connected
              @param retry_kwargs: keyword args for L{retry}
              """
              for c in self.config.sections('connector'):
      >           assert retry(lambda: self.is_connected(port=c['port'], host=self.get_host(c.get('protocolFamily'))),
                               **retry_kwargs), "Port not connected %s" % c['port']
      E           AssertionError: Port not connected 25519
      E           assert None
      E            +  where None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: AssertionError
      _____ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_03_server_reconnect _____
      
      cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'>
      name = 'EA2', mode = 'edge'
      extra = [('connector', {'host': '127.0.0.1', 'idleTimeoutSeconds': '120', 'name': 'uplink', 'port': 25519, ...}), ('httpConnec...protocolVersion': 'HTTP1'}), ('httpConnector', {'address': 'testServer10', 'port': 25522, 'protocolVersion': 'HTTP1'})]
      
          @classmethod
          def router(cls, name, mode, extra):
              config = [
                  ('router', {'mode': mode,
                              'id': name,
                              'allowUnsettledMulticast': 'yes'}),
                  ('listener', {'role': 'normal',
                                'port': cls.tester.get_port()}),
                  ('address', {'prefix': 'closest', 'distribution': 'closest'}),
                  ('address', {'prefix': 'multicast', 'distribution': 'multicast'}),
              ]
          
              if extra:
                  config.extend(extra)
              config = Qdrouterd.Config(config)
      >       cls.routers.append(cls.tester.qdrouterd(name, config, wait=True))
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: in qdrouterd
          return self.cleanup(Qdrouterd(*args, **kwargs))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: in __init__
          self.wait_ready()
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: in wait_ready
          self.wait_connectors(**retry_kwargs)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {}
      @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>
      @py_assert4 = None
      @py_format6 = 'Port not connected 25519\n>assert None\n{None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})\n}'
      
          def wait_connectors(self, **retry_kwargs):
              """
              Wait for all connectors to be connected
              @param retry_kwargs: keyword args for L{retry}
              """
              for c in self.config.sections('connector'):
      >           assert retry(lambda: self.is_connected(port=c['port'], host=self.get_host(c.get('protocolFamily'))),
                               **retry_kwargs), "Port not connected %s" % c['port']
      E           AssertionError: Port not connected 25519
      E           assert None
      E            +  where None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: AssertionError
      _ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_04_server_pining_for_the_fjords _
      
      cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'>
      name = 'EA2', mode = 'edge'
      extra = [('connector', {'host': '127.0.0.1', 'idleTimeoutSeconds': '120', 'name': 'uplink', 'port': 25519, ...}), ('httpConnec...protocolVersion': 'HTTP1'}), ('httpConnector', {'address': 'testServer10', 'port': 25522, 'protocolVersion': 'HTTP1'})]
      
          @classmethod
          def router(cls, name, mode, extra):
              config = [
                  ('router', {'mode': mode,
                              'id': name,
                              'allowUnsettledMulticast': 'yes'}),
                  ('listener', {'role': 'normal',
                                'port': cls.tester.get_port()}),
                  ('address', {'prefix': 'closest', 'distribution': 'closest'}),
                  ('address', {'prefix': 'multicast', 'distribution': 'multicast'}),
              ]
          
              if extra:
                  config.extend(extra)
              config = Qdrouterd.Config(config)
      >       cls.routers.append(cls.tester.qdrouterd(name, config, wait=True))
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: in qdrouterd
          return self.cleanup(Qdrouterd(*args, **kwargs))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: in __init__
          self.wait_ready()
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: in wait_ready
          self.wait_connectors(**retry_kwargs)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {}
      @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>
      @py_assert4 = None
      @py_format6 = 'Port not connected 25519\n>assert None\n{None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})\n}'
      
          def wait_connectors(self, **retry_kwargs):
              """
              Wait for all connectors to be connected
              @param retry_kwargs: keyword args for L{retry}
              """
              for c in self.config.sections('connector'):
      >           assert retry(lambda: self.is_connected(port=c['port'], host=self.get_host(c.get('protocolFamily'))),
                               **retry_kwargs), "Port not connected %s" % c['port']
      E           AssertionError: Port not connected 25519
      E           assert None
      E            +  where None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: AssertionError
      ___ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_05_large_streaming_msg ____
      
      cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'>
      name = 'EA2', mode = 'edge'
      extra = [('connector', {'host': '127.0.0.1', 'idleTimeoutSeconds': '120', 'name': 'uplink', 'port': 25519, ...}), ('httpConnec...protocolVersion': 'HTTP1'}), ('httpConnector', {'address': 'testServer10', 'port': 25522, 'protocolVersion': 'HTTP1'})]
      
          @classmethod
          def router(cls, name, mode, extra):
              config = [
                  ('router', {'mode': mode,
                              'id': name,
                              'allowUnsettledMulticast': 'yes'}),
                  ('listener', {'role': 'normal',
                                'port': cls.tester.get_port()}),
                  ('address', {'prefix': 'closest', 'distribution': 'closest'}),
                  ('address', {'prefix': 'multicast', 'distribution': 'multicast'}),
              ]
          
              if extra:
                  config.extend(extra)
              config = Qdrouterd.Config(config)
      >       cls.routers.append(cls.tester.qdrouterd(name, config, wait=True))
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: in qdrouterd
          return self.cleanup(Qdrouterd(*args, **kwargs))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: in __init__
          self.wait_ready()
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: in wait_ready
          self.wait_connectors(**retry_kwargs)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {}
      @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>
      @py_assert4 = None
      @py_format6 = 'Port not connected 25519\n>assert None\n{None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})\n}'
      
          def wait_connectors(self, **retry_kwargs):
              """
              Wait for all connectors to be connected
              @param retry_kwargs: keyword args for L{retry}
              """
              for c in self.config.sections('connector'):
      >           assert retry(lambda: self.is_connected(port=c['port'], host=self.get_host(c.get('protocolFamily'))),
                               **retry_kwargs), "Port not connected %s" % c['port']
      E           AssertionError: Port not connected 25519
      E           assert None
      E            +  where None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: AssertionError
      __ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_1001_client_request_close __
      
      cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'>
      
          @classmethod
          def setUpClass(cls):
              """Start a router"""
              super(Http1AdaptorEdge2EdgeTest, cls).setUpClass()
          
              # configuration:
              # one edge, one interior
              #
              #  +-------+    +---------+    +-------+
              #  |  EA1  |<==>|  INT.A  |<==>|  EA2  |
              #  +-------+    +---------+    +-------+
              #      ^                           ^
              #      |                           |
              #      V                           V
              #  <clients>                   <servers>
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('INT.A', 'interior', [('listener', {'role': 'edge', 'port': cls.INTA_edge1_port}),
                                               ('listener', {'role': 'edge', 'port': cls.INTA_edge2_port}),
                                               ])
              cls.INT_A = cls.routers[0]
              cls.INT_A.listener = cls.INT_A.addresses[0]
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('EA1', 'edge',
                         [('connector', {'name': 'uplink', 'role': 'edge',
                                         'port': cls.INTA_edge1_port}),
                          ('httpListener', {'port': cls.http_listener11_port,
                                            'protocolVersion': 'HTTP1',
                                            'address': 'testServer11'}),
                          ('httpListener', {'port': cls.http_listener10_port,
                                            'protocolVersion': 'HTTP1',
                                            'address': 'testServer10'})
                          ])
              cls.EA1 = cls.routers[1]
              cls.EA1.listener = cls.EA1.addresses[0]
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('EA2', 'edge',
                         [('connector', {'name': 'uplink', 'role': 'edge',
                                         'port': cls.INTA_edge2_port}),
                          ('httpConnector', {'port': cls.http_server11_port,
                                             'protocolVersion': 'HTTP1',
                                             'address': 'testServer11'}),
                          ('httpConnector', {'port': cls.http_server10_port,
                                             'protocolVersion': 'HTTP1',
      >                                      'address': 'testServer10'})
                          ])
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:391: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: in router
          cls.routers.append(cls.tester.qdrouterd(name, config, wait=True))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: in qdrouterd
          return self.cleanup(Qdrouterd(*args, **kwargs))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: in __init__
          self.wait_ready()
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: in wait_ready
          self.wait_connectors(**retry_kwargs)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {}
      @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>
      @py_assert4 = None
      @py_format6 = 'Port not connected 25519\n>assert None\n{None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})\n}'
      
          def wait_connectors(self, **retry_kwargs):
              """
              Wait for all connectors to be connected
              @param retry_kwargs: keyword args for L{retry}
              """
              for c in self.config.sections('connector'):
      >           assert retry(lambda: self.is_connected(port=c['port'], host=self.get_host(c.get('protocolFamily'))),
                               **retry_kwargs), "Port not connected %s" % c['port']
      E           AssertionError: Port not connected 25519
      E           assert None
      E            +  where None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: AssertionError
      _ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_1002_client_response_close __
      
      cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'>
      
          @classmethod
          def setUpClass(cls):
              """Start a router"""
              super(Http1AdaptorEdge2EdgeTest, cls).setUpClass()
          
              # configuration:
              # one edge, one interior
              #
              #  +-------+    +---------+    +-------+
              #  |  EA1  |<==>|  INT.A  |<==>|  EA2  |
              #  +-------+    +---------+    +-------+
              #      ^                           ^
              #      |                           |
              #      V                           V
              #  <clients>                   <servers>
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('INT.A', 'interior', [('listener', {'role': 'edge', 'port': cls.INTA_edge1_port}),
                                               ('listener', {'role': 'edge', 'port': cls.INTA_edge2_port}),
                                               ])
              cls.INT_A = cls.routers[0]
              cls.INT_A.listener = cls.INT_A.addresses[0]
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('EA1', 'edge',
                         [('connector', {'name': 'uplink', 'role': 'edge',
                                         'port': cls.INTA_edge1_port}),
                          ('httpListener', {'port': cls.http_listener11_port,
                                            'protocolVersion': 'HTTP1',
                                            'address': 'testServer11'}),
                          ('httpListener', {'port': cls.http_listener10_port,
                                            'protocolVersion': 'HTTP1',
                                            'address': 'testServer10'})
                          ])
              cls.EA1 = cls.routers[1]
              cls.EA1.listener = cls.EA1.addresses[0]
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('EA2', 'edge',
                         [('connector', {'name': 'uplink', 'role': 'edge',
                                         'port': cls.INTA_edge2_port}),
                          ('httpConnector', {'port': cls.http_server11_port,
                                             'protocolVersion': 'HTTP1',
                                             'address': 'testServer11'}),
                          ('httpConnector', {'port': cls.http_server10_port,
                                             'protocolVersion': 'HTTP1',
      >                                      'address': 'testServer10'})
                          ])
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:391: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: in router
          cls.routers.append(cls.tester.qdrouterd(name, config, wait=True))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: in qdrouterd
          return self.cleanup(Qdrouterd(*args, **kwargs))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: in __init__
          self.wait_ready()
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: in wait_ready
          self.wait_connectors(**retry_kwargs)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {}
      @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>
      @py_assert4 = None
      @py_format6 = 'Port not connected 25519\n>assert None\n{None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})\n}'
      
          def wait_connectors(self, **retry_kwargs):
              """
              Wait for all connectors to be connected
              @param retry_kwargs: keyword args for L{retry}
              """
              for c in self.config.sections('connector'):
      >           assert retry(lambda: self.is_connected(port=c['port'], host=self.get_host(c.get('protocolFamily'))),
                               **retry_kwargs), "Port not connected %s" % c['port']
      E           AssertionError: Port not connected 25519
      E           assert None
      E            +  where None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: AssertionError
      ________ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_2000_curl_get ________
      
      cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'>
      
          @classmethod
          def setUpClass(cls):
              """Start a router"""
              super(Http1AdaptorEdge2EdgeTest, cls).setUpClass()
          
              # configuration:
              # one edge, one interior
              #
              #  +-------+    +---------+    +-------+
              #  |  EA1  |<==>|  INT.A  |<==>|  EA2  |
              #  +-------+    +---------+    +-------+
              #      ^                           ^
              #      |                           |
              #      V                           V
              #  <clients>                   <servers>
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('INT.A', 'interior', [('listener', {'role': 'edge', 'port': cls.INTA_edge1_port}),
                                               ('listener', {'role': 'edge', 'port': cls.INTA_edge2_port}),
                                               ])
              cls.INT_A = cls.routers[0]
              cls.INT_A.listener = cls.INT_A.addresses[0]
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('EA1', 'edge',
                         [('connector', {'name': 'uplink', 'role': 'edge',
                                         'port': cls.INTA_edge1_port}),
                          ('httpListener', {'port': cls.http_listener11_port,
                                            'protocolVersion': 'HTTP1',
                                            'address': 'testServer11'}),
                          ('httpListener', {'port': cls.http_listener10_port,
                                            'protocolVersion': 'HTTP1',
                                            'address': 'testServer10'})
                          ])
              cls.EA1 = cls.routers[1]
              cls.EA1.listener = cls.EA1.addresses[0]
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('EA2', 'edge',
                         [('connector', {'name': 'uplink', 'role': 'edge',
                                         'port': cls.INTA_edge2_port}),
                          ('httpConnector', {'port': cls.http_server11_port,
                                             'protocolVersion': 'HTTP1',
                                             'address': 'testServer11'}),
                          ('httpConnector', {'port': cls.http_server10_port,
                                             'protocolVersion': 'HTTP1',
      >                                      'address': 'testServer10'})
                          ])
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:391: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: in router
          cls.routers.append(cls.tester.qdrouterd(name, config, wait=True))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: in qdrouterd
          return self.cleanup(Qdrouterd(*args, **kwargs))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: in __init__
          self.wait_ready()
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: in wait_ready
          self.wait_connectors(**retry_kwargs)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {}
      @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>
      @py_assert4 = None
      @py_format6 = 'Port not connected 25519\n>assert None\n{None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})\n}'
      
          def wait_connectors(self, **retry_kwargs):
              """
              Wait for all connectors to be connected
              @param retry_kwargs: keyword args for L{retry}
              """
              for c in self.config.sections('connector'):
      >           assert retry(lambda: self.is_connected(port=c['port'], host=self.get_host(c.get('protocolFamily'))),
                               **retry_kwargs), "Port not connected %s" % c['port']
      E           AssertionError: Port not connected 25519
      E           assert None
      E            +  where None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: AssertionError
      ________ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_2001_curl_put ________
      
      cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'>
      
          @classmethod
          def setUpClass(cls):
              """Start a router"""
              super(Http1AdaptorEdge2EdgeTest, cls).setUpClass()
          
              # configuration:
              # one edge, one interior
              #
              #  +-------+    +---------+    +-------+
              #  |  EA1  |<==>|  INT.A  |<==>|  EA2  |
              #  +-------+    +---------+    +-------+
              #      ^                           ^
              #      |                           |
              #      V                           V
              #  <clients>                   <servers>
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('INT.A', 'interior', [('listener', {'role': 'edge', 'port': cls.INTA_edge1_port}),
                                               ('listener', {'role': 'edge', 'port': cls.INTA_edge2_port}),
                                               ])
              cls.INT_A = cls.routers[0]
              cls.INT_A.listener = cls.INT_A.addresses[0]
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('EA1', 'edge',
                         [('connector', {'name': 'uplink', 'role': 'edge',
                                         'port': cls.INTA_edge1_port}),
                          ('httpListener', {'port': cls.http_listener11_port,
                                            'protocolVersion': 'HTTP1',
                                            'address': 'testServer11'}),
                          ('httpListener', {'port': cls.http_listener10_port,
                                            'protocolVersion': 'HTTP1',
                                            'address': 'testServer10'})
                          ])
              cls.EA1 = cls.routers[1]
              cls.EA1.listener = cls.EA1.addresses[0]
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('EA2', 'edge',
                         [('connector', {'name': 'uplink', 'role': 'edge',
                                         'port': cls.INTA_edge2_port}),
                          ('httpConnector', {'port': cls.http_server11_port,
                                             'protocolVersion': 'HTTP1',
                                             'address': 'testServer11'}),
                          ('httpConnector', {'port': cls.http_server10_port,
                                             'protocolVersion': 'HTTP1',
      >                                      'address': 'testServer10'})
                          ])
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:391: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: in router
          cls.routers.append(cls.tester.qdrouterd(name, config, wait=True))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: in qdrouterd
          return self.cleanup(Qdrouterd(*args, **kwargs))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: in __init__
          self.wait_ready()
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: in wait_ready
          self.wait_connectors(**retry_kwargs)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {}
      @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>
      @py_assert4 = None
      @py_format6 = 'Port not connected 25519\n>assert None\n{None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})\n}'
      
          def wait_connectors(self, **retry_kwargs):
              """
              Wait for all connectors to be connected
              @param retry_kwargs: keyword args for L{retry}
              """
              for c in self.config.sections('connector'):
      >           assert retry(lambda: self.is_connected(port=c['port'], host=self.get_host(c.get('protocolFamily'))),
                               **retry_kwargs), "Port not connected %s" % c['port']
      E           AssertionError: Port not connected 25519
      E           assert None
      E            +  where None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: AssertionError
      _______ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_2002_curl_post ________
      
      cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'>
      
          @classmethod
          def setUpClass(cls):
              """Start a router"""
              super(Http1AdaptorEdge2EdgeTest, cls).setUpClass()
          
              # configuration:
              # one edge, one interior
              #
              #  +-------+    +---------+    +-------+
              #  |  EA1  |<==>|  INT.A  |<==>|  EA2  |
              #  +-------+    +---------+    +-------+
              #      ^                           ^
              #      |                           |
              #      V                           V
              #  <clients>                   <servers>
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('INT.A', 'interior', [('listener', {'role': 'edge', 'port': cls.INTA_edge1_port}),
                                               ('listener', {'role': 'edge', 'port': cls.INTA_edge2_port}),
                                               ])
              cls.INT_A = cls.routers[0]
              cls.INT_A.listener = cls.INT_A.addresses[0]
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('EA1', 'edge',
                         [('connector', {'name': 'uplink', 'role': 'edge',
                                         'port': cls.INTA_edge1_port}),
                          ('httpListener', {'port': cls.http_listener11_port,
                                            'protocolVersion': 'HTTP1',
                                            'address': 'testServer11'}),
                          ('httpListener', {'port': cls.http_listener10_port,
                                            'protocolVersion': 'HTTP1',
                                            'address': 'testServer10'})
                          ])
              cls.EA1 = cls.routers[1]
              cls.EA1.listener = cls.EA1.addresses[0]
          
              super(Http1AdaptorEdge2EdgeTest, cls).\
                  router('EA2', 'edge',
                         [('connector', {'name': 'uplink', 'role': 'edge',
                                         'port': cls.INTA_edge2_port}),
                          ('httpConnector', {'port': cls.http_server11_port,
                                             'protocolVersion': 'HTTP1',
                                             'address': 'testServer11'}),
                          ('httpConnector', {'port': cls.http_server10_port,
                                             'protocolVersion': 'HTTP1',
      >                                      'address': 'testServer10'})
                          ])
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:391: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: in router
          cls.routers.append(cls.tester.qdrouterd(name, config, wait=True))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: in qdrouterd
          return self.cleanup(Qdrouterd(*args, **kwargs))
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: in __init__
          self.wait_ready()
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: in wait_ready
          self.wait_connectors(**retry_kwargs)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {}
      @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>
      @py_assert4 = None
      @py_format6 = 'Port not connected 25519\n>assert None\n{None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})\n}'
      
          def wait_connectors(self, **retry_kwargs):
              """
              Wait for all connectors to be connected
              @param retry_kwargs: keyword args for L{retry}
              """
              for c in self.config.sections('connector'):
      >           assert retry(lambda: self.is_connected(port=c['port'], host=self.get_host(c.get('protocolFamily'))),
                               **retry_kwargs), "Port not connected %s" % c['port']
      E           AssertionError: Port not connected 25519
      E           assert None
      E            +  where None = retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{})
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: AssertionError
      =================================== FAILURES ===================================
      _______________ Http1AdaptorManagementTest.test_01_create_delete _______________
      
      self = <system_tests_http1_adaptor.Http1AdaptorManagementTest testMethod=test_01_create_delete>
      
          def test_01_create_delete(self):
              """ Create and delete HTTP1 connectors and listeners.  The
              connectors/listeners are created on the edge router.  Verify that the
              adaptor properly notifies the interior of the subscribers/producers.
              """
              e_mgmt = self.e_router.management
              self.assertEqual(0, len(e_mgmt.query(type=self.LISTENER_TYPE).results))
              self.assertEqual(0, len(e_mgmt.query(type=self.CONNECTOR_TYPE).results))
          
              e_mgmt.create(type=self.CONNECTOR_TYPE,
                            name="ServerConnector",
                            attributes={'address': 'closest/http1Service',
                                        'port': self.http_server_port,
                                        'protocolVersion': 'HTTP1'})
          
              e_mgmt.create(type=self.LISTENER_TYPE,
                            name="ClientListener",
                            attributes={'address': 'closest/http1Service',
                                        'port': self.http_listener_port,
                                        'protocolVersion': 'HTTP1'})
          
              # verify the entities have been created and http traffic works
          
              self.assertEqual(1, len(e_mgmt.query(type=self.LISTENER_TYPE).results))
              self.assertEqual(1, len(e_mgmt.query(type=self.CONNECTOR_TYPE).results))
          
              count, error = http1_ping(sport=self.http_server_port,
                                        cport=self.http_listener_port)
              self.assertIsNone(error)
              self.assertEqual(1, count)
          
              # now check the interior router for the closest/http1Service address
              self.i_router.wait_address("closest/http1Service", subscribers=1)
          
              #
              # delete the connector and listener; wait for the associated connection
              # to be removed
              #
              e_mgmt.delete(type=self.CONNECTOR_TYPE, name="ServerConnector")
              self.assertEqual(0, len(e_mgmt.query(type=self.CONNECTOR_TYPE).results))
              e_mgmt.delete(type=self.LISTENER_TYPE, name="ClientListener")
              self.assertEqual(0, len(e_mgmt.query(type=self.LISTENER_TYPE).results))
          
              # will hit test timeout on failure:
              while True:
                  hconns = 0
                  obj = e_mgmt.query(type=self.CONNECTION_TYPE,
                                     attribute_names=["protocol"])
                  for item in obj.get_dicts():
                      if "http/1.x" in item["protocol"]:
                          hconns += 1
                  if hconns == 0:
                      break
                  sleep(0.25)
          
              # When a connector is configured the router will periodically attempt
              # to connect to the server address. To prove that the connector has
              # been completely removed listen for connection attempts on the server
              # port.
              s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
              s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
              s.bind(("", self.http_server_port))
              s.setblocking(True)
              s.settimeout(3)  # reconnect attempts every 2.5 seconds
              s.listen(1)
              with self.assertRaises(socket.timeout):
      >           conn, addr = s.accept()
      E           AssertionError: timeout not raised
      
      /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:158: AssertionError
      __________ Http1AdaptorManagementTest.test_01_delete_active_connector __________
      
      self = <system_tests_http1_adaptor.Http1AdaptorManagementTest testMethod=test_01_delete_active_connector>
      
          def test_01_delete_active_connector(self):
              """Delete an HTTP1 connector that is currently connected to a server.
              Verify the connection is dropped.
              """
              e_mgmt = self.e_router.management
              self.assertEqual(0, len(e_mgmt.query(type=self.CONNECTOR_TYPE).results))
          
              e_mgmt.create(type=self.CONNECTOR_TYPE,
                            name="ServerConnector",
                            attributes={'address': 'closest/http1Service',
                                        'port': self.http_server_port,
                                        'protocolVersion': 'HTTP1'})
          
              # verify the connector has been created and attach a dummy server
              self.assertEqual(1, len(e_mgmt.query(type=self.CONNECTOR_TYPE).results))
          
              server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
              server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
      >       server.bind(("", self.http_server_port))
      E       OSError: [Errno 98] Address already in use
      

      One scenario against which the tests are not protecting is that something without SO_REUSEADDR was listening on the socket before. In that case, attempt to connect will fail (nothing is there listening now) but a new bind will fail too (we are lingering after use). The port range dispatch is using is chosen not to fall into

      $ cat /proc/sys/net/ipv4/ip_local_port_range
      32768   60999
      

      so we should be safe in that respect (local randomly assigned ports don't clash). Still, it seems sensible to me to try checking a bind() before we hand over the port.

      Attachments

        Issue Links

          Activity

            People

              jdanek Jiri Daněk
              jdanek Jiri Daněk
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: