Qpid
  1. Qpid
  2. QPID-4872

python client occasionally fails with "[Errno 1] _ssl.c:1217: error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry" error

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20
    • Fix Version/s: 0.23
    • Component/s: Python Client
    • Labels:
      None
    1. bz950501.py
      0.7 kB
      Ken Giusti

      Issue Links

        Activity

        Ken Giusti created issue -
        Ken Giusti made changes -
        Ken Giusti made changes -
        Link This issue is related to QPID-3175 [ QPID-3175 ]
        Hide
        Ken Giusti added a comment -
        Show
        Ken Giusti added a comment - Originally reported by Michal Zerola https://issues.apache.org/jira/secure/ViewProfile.jspa?name=zer0
        Hide
        Ken Giusti added a comment -

        Reproducer. I have to run two or three of these in parallel before the exception is raised.

        I also run "drain -f ken > /dev/null" to keep the queue from filling.

        Show
        Ken Giusti added a comment - Reproducer. I have to run two or three of these in parallel before the exception is raised. I also run "drain -f ken > /dev/null" to keep the queue from filling.
        Ken Giusti made changes -
        Attachment bz950501.py [ 12584014 ]
        Hide
        Ken Giusti added a comment -

        Root cause appears to be due to this python bug:

        http://bugs.python.org/issue8240

        Been around for awhile - probably not going to be fixed in python anytime soon.

        Need to work-around this bug.

        Show
        Ken Giusti added a comment - Root cause appears to be due to this python bug: http://bugs.python.org/issue8240 Been around for awhile - probably not going to be fixed in python anytime soon. Need to work-around this bug.
        Hide
        Jakub Scholz added a comment -

        Hi Ken,

        Is there some reason why not to simply create the SSL socket as blocking? For us it definitely fixed the error which was occurring quite often during our continuous integration runs. And I do not assume it would cause any problems if the non-SSL socket is already blocking. Or do I miss something?

        Regards
        Jakub

        Show
        Jakub Scholz added a comment - Hi Ken, Is there some reason why not to simply create the SSL socket as blocking? For us it definitely fixed the error which was occurring quite often during our continuous integration runs. And I do not assume it would cause any problems if the non-SSL socket is already blocking. Or do I miss something? Regards Jakub
        Hide
        Ken Giusti added a comment -

        Hi Jakub, you're correct in that configuring the socket to use blocking mode will prevent this error. However I suspect that it could cause problems if a timeout is passed to the send() call. I think that using blocking could end up having send() block and not return within the requested timeout.

        Although the non-SSL socket blocks, the python client uses select() in order to avoid making blocking calls to the socket. So even though the socket is blocking, the code won't make a call to the socket unless it knows it will not block.

        This won't work for SSL sockets. This is because our application cannot control when the SSL library writes or reads from the socket. Specifically, if a read() (or write()) is done on a blocking SSL socket, the underlying SSL library may in turn call write() (or read()) as needed in order to perform an SSL handshake. For example, if the python client's select() indicates that the socket is writable (but not readable), and we call write() on the SSL socket, the SSL library may then need to perform a read(), which could block indefinitely. Depending on how long SSL blocks, there's no guarantee that it will return in enough time to handle the timeout properly.

        Show
        Ken Giusti added a comment - Hi Jakub, you're correct in that configuring the socket to use blocking mode will prevent this error. However I suspect that it could cause problems if a timeout is passed to the send() call. I think that using blocking could end up having send() block and not return within the requested timeout. Although the non-SSL socket blocks, the python client uses select() in order to avoid making blocking calls to the socket. So even though the socket is blocking, the code won't make a call to the socket unless it knows it will not block. This won't work for SSL sockets. This is because our application cannot control when the SSL library writes or reads from the socket. Specifically, if a read() (or write()) is done on a blocking SSL socket, the underlying SSL library may in turn call write() (or read()) as needed in order to perform an SSL handshake. For example, if the python client's select() indicates that the socket is writable (but not readable), and we call write() on the SSL socket, the SSL library may then need to perform a read(), which could block indefinitely. Depending on how long SSL blocks, there's no guarantee that it will return in enough time to handle the timeout properly.
        Hide
        Jakub Scholz added a comment -

        Oki, understand. Thanks for the explanation.

        Regards
        Jakub

        Show
        Jakub Scholz added a comment - Oki, understand. Thanks for the explanation. Regards Jakub
        Hide
        Ken Giusti added a comment -

        Reviewboard entry for this issue, which includes the proposed patch:

        https://reviews.apache.org/r/11319/

        Show
        Ken Giusti added a comment - Reviewboard entry for this issue, which includes the proposed patch: https://reviews.apache.org/r/11319/
        Show
        Ken Giusti added a comment - Fix: http://svn.apache.org/viewvc?view=revision&revision=1485331
        Ken Giusti made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Justin Ross added a comment -
        Show
        Justin Ross added a comment - Released in Qpid 0.24, http://qpid.apache.org/releases/qpid-0.24/index.html
        Justin Ross made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        ASF subversion and git services added a comment -

        Commit 1598353 from Ken Giusti in branch 'qpid/trunk'
        [ https://svn.apache.org/r1598353 ]

        QPID-5773: back out non-functional recv path change from QPID-4872

        Show
        ASF subversion and git services added a comment - Commit 1598353 from Ken Giusti in branch 'qpid/trunk' [ https://svn.apache.org/r1598353 ] QPID-5773 : back out non-functional recv path change from QPID-4872
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        1d 4h 15m 1 Ken Giusti 22/May/13 19:49
        Resolved Resolved Closed Closed
        108d 18h 48m 1 Justin Ross 08/Sep/13 14:37

          People

          • Assignee:
            Ken Giusti
            Reporter:
            Ken Giusti
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development