Uploaded image for project: 'Qpid'
  1. Qpid
  2. QPID-3237

Python broker tests hang in environments where the available file handles > 1024

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Auto Closed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Python Test Suite
    • Labels:
      None
    • Environment:

      As root, append to the /etc/security/limits.conf the following line to assign yourself the ability to open 64k file handles (substitute your login id):

      kpvdr - nofile 65536

      then log out/in as yourself.

      Description

      When the environment is modified to allow the qpid user to open more than 1024 file handles, then the following python tests fail/hang. The underlying cause seems to be the use of the python call to select(), which uses the underlying unix ::select() call. However, it is a well-known limitation that the unix select() cannot handle more than 1024 file handles. Perhaps switching the tests to using epoll would solve this issue.

      qpid.tests.messaging.endpoints.SetupTests.testOpenCloseResourceLeaks ................................................................................................. start
      Exception in thread Thread-81:
      Traceback (most recent call last):
      File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner
      self.run()
      File "/usr/lib64/python2.6/threading.py", line 484, in run
      self._target(*self.args, **self._kwargs)
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/selector.py", line 119, in run
      rd, wr, ex = select(self.reading, self.writing, (), timeout)
      ValueError: filedescriptor out of range in select()

      qpid.tests.messaging.endpoints.SetupTests.testOpenCloseResourceLeaks ................................................................................................. fail
      Error during test:
      Traceback (most recent call last):
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/commands/qpid-python-test", line 311, in run
      phase()
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/tests/messaging/endpoints.py", line 86, in testOpenCloseResourceLeaks
      conn = Connection.establish(self.broker, **self.connection_options())
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/messaging/endpoints.py", line 68, in establish
      conn.open()
      File "<string>", line 6, in open
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/messaging/endpoints.py", line 244, in open
      self.attach()
      File "<string>", line 6, in attach
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/messaging/endpoints.py", line 262, in attach
      self._ewait(lambda: self._transport_connected and not self._unlinked())
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/messaging/endpoints.py", line 196, in _ewait
      result = self._wait(lambda: self.error or predicate(), timeout)
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/messaging/endpoints.py", line 181, in _wait
      return self._waiter.wait(predicate, timeout=timeout)
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/concurrency.py", line 57, in wait
      self.condition.wait(3)
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/concurrency.py", line 96, in wait
      sw.wait(timeout)
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/compat.py", line 53, in wait
      ready, _, _ = select([self], [], [], timeout)
      ValueError: filedescriptor out of range in select()
      qpid.tests.messaging.endpoints.SetupTests.testOpenFailResourceLeaks .................................................................................................. fail
      Error during test:
      Traceback (most recent call last):
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/commands/qpid-python-test", line 311, in run
      phase()
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/tests/messaging/endpoints.py", line 102, in testOpenFailResourceLeaks
      conn._wait(lambda: False, timeout=0.001)
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/messaging/endpoints.py", line 181, in _wait
      return self._waiter.wait(predicate, timeout=timeout)
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/concurrency.py", line 59, in wait
      self.condition.wait(timeout - passed)
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/concurrency.py", line 96, in wait
      sw.wait(timeout)
      File "/home/kpvdr/mrg/qpid/cpp/src/tests/python/qpid/compat.py", line 53, in wait
      ready, _, _ = select([self], [], [], timeout)
      ValueError: filedescriptor out of range in select()
      qpid.tests.messaging.endpoints.SetupTests.testOpenReconnectURLs ......................................................................................................<hang>

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              kpvdr Kim van der Riet
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: