Uploaded image for project: 'Thrift'
  1. Thrift
  2. THRIFT-2918

Race condition in Python TProcessPoolServer test

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.3
    • Component/s: Python - Library
    • Labels:
      None
    • Environment:

      openSUSE 13.2
      Python 2.7.8 (default, Sep 30 2014, 15:34:38) [GCC] on linux2

      Description

      make check gets stuck very reproducible in my VM at Test run #218:

      Test run #217:  (includes gen-py-default) Server=TProcessPoolServer,  Proto=accel,  zlib=False,  SSL=False
      Testing server TProcessPoolServer: /usr/bin/python ./TestServer.py --genpydir=gen-py-default --protocol=accel --port=9090 TProcessPoolServer
      Testing client: /usr/bin/python ./TestClient.py --genpydir=gen-py-default --protocol=accel --port=9090 --transport=buffered
      ...testException(Safe)
      testException(Xception)
      testException(throw_undeclared)
      ...............
      ----------------------------------------------------------------------
      Ran 18 tests in 0.563s
      
      OK
      Giving TProcessPoolServer (proto=accel,zlib=False,ssl=False) an extra 3 seconds for childprocesses to terminate via alarm
      Terminating worker: <Process(Process-1, started daemon)>
      Terminating worker: <Process(Process-2, started daemon)>
      Terminating worker: <Process(Process-3, started daemon)>
      Terminating worker: <Process(Process-4, started daemon)>
      Terminating worker: <Process(Process-5, started daemon)>
      Requesting server to stop()
      OK: Finished (includes gen-py-default)  TProcessPoolServer / accel proto / zlib=False / SSL=False.   217 combinations tested.
      
      Test run #218:  (includes gen-py-default) Server=TProcessPoolServer,  Proto=accel,  zlib=False,  SSL=True
      Testing server TProcessPoolServer: /usr/bin/python ./TestServer.py --genpydir=gen-py-default --protocol=accel --port=9090 --ssl TProcessPoolServer
      Testing client: /usr/bin/python ./TestClient.py --genpydir=gen-py-default --protocol=accel --port=9090 --ssl --transport=buffered
      ...testException(Safe)
      testException(Xception)
      testException(throw_undeclared)
      ..........Terminating worker: <Process(Process-1, started daemon)>
      Terminating worker: <Process(Process-2, started daemon)>
      Terminating worker: <Process(Process-3, started daemon)>
      Terminating worker: <Process(Process-4, started daemon)>
      Terminating worker: <Process(Process-5, started daemon)>
      Requesting server to stop()
      

      After fiddling a bit around with it I got it to work by increasing the alarm() timeout from 2 seconds to 4 seconds.

      I'm not a Python expert, but the code looks somewhat interesting to me:

      • The server code starts the workers, but some piece of code outside of the server is responsible for terminating them. Is that really idiomatic in Python or just bad design?
      • The Condition() object used in TProcessPoolsServer.py should probably be replaced by an Event() object. Especially, as Condition.wait() seems to have it's own perils and Event is much easier to use.
      • Calling Condition.aquire() without a matching release() within a while True: loop looks also not very convincing to me. AFAIK the second call to aquire() will block, if that ever happens (it did not in my tests).

      The bad news is, that neither of the proposed changes above had any effect on the race conditions, except increasing the timeout - but that is merely a workaround, not a solution.

        Attachments

          Activity

            People

            • Assignee:
              jensg Jens Geyer
              Reporter:
              jensg Jens Geyer
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: