Qpid
  1. Qpid
  2. QPID-4872

python client occasionally fails with "[Errno 1] _ssl.c:1217: error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry" error

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20
    • Fix Version/s: 0.23
    • Component/s: Python Client
    • Labels:
      None
    1. bz950501.py
      0.7 kB
      Ken Giusti

      Issue Links

        Activity

        Hide
        ASF subversion and git services added a comment -

        Commit 1598353 from Ken Giusti in branch 'qpid/trunk'
        [ https://svn.apache.org/r1598353 ]

        QPID-5773: back out non-functional recv path change from QPID-4872

        Show
        ASF subversion and git services added a comment - Commit 1598353 from Ken Giusti in branch 'qpid/trunk' [ https://svn.apache.org/r1598353 ] QPID-5773 : back out non-functional recv path change from QPID-4872
        Hide
        Justin Ross added a comment -
        Show
        Justin Ross added a comment - Released in Qpid 0.24, http://qpid.apache.org/releases/qpid-0.24/index.html
        Show
        Ken Giusti added a comment - Fix: http://svn.apache.org/viewvc?view=revision&revision=1485331
        Hide
        Ken Giusti added a comment -

        Reviewboard entry for this issue, which includes the proposed patch:

        https://reviews.apache.org/r/11319/

        Show
        Ken Giusti added a comment - Reviewboard entry for this issue, which includes the proposed patch: https://reviews.apache.org/r/11319/
        Hide
        Jakub Scholz added a comment -

        Oki, understand. Thanks for the explanation.

        Regards
        Jakub

        Show
        Jakub Scholz added a comment - Oki, understand. Thanks for the explanation. Regards Jakub
        Hide
        Ken Giusti added a comment -

        Hi Jakub, you're correct in that configuring the socket to use blocking mode will prevent this error. However I suspect that it could cause problems if a timeout is passed to the send() call. I think that using blocking could end up having send() block and not return within the requested timeout.

        Although the non-SSL socket blocks, the python client uses select() in order to avoid making blocking calls to the socket. So even though the socket is blocking, the code won't make a call to the socket unless it knows it will not block.

        This won't work for SSL sockets. This is because our application cannot control when the SSL library writes or reads from the socket. Specifically, if a read() (or write()) is done on a blocking SSL socket, the underlying SSL library may in turn call write() (or read()) as needed in order to perform an SSL handshake. For example, if the python client's select() indicates that the socket is writable (but not readable), and we call write() on the SSL socket, the SSL library may then need to perform a read(), which could block indefinitely. Depending on how long SSL blocks, there's no guarantee that it will return in enough time to handle the timeout properly.

        Show
        Ken Giusti added a comment - Hi Jakub, you're correct in that configuring the socket to use blocking mode will prevent this error. However I suspect that it could cause problems if a timeout is passed to the send() call. I think that using blocking could end up having send() block and not return within the requested timeout. Although the non-SSL socket blocks, the python client uses select() in order to avoid making blocking calls to the socket. So even though the socket is blocking, the code won't make a call to the socket unless it knows it will not block. This won't work for SSL sockets. This is because our application cannot control when the SSL library writes or reads from the socket. Specifically, if a read() (or write()) is done on a blocking SSL socket, the underlying SSL library may in turn call write() (or read()) as needed in order to perform an SSL handshake. For example, if the python client's select() indicates that the socket is writable (but not readable), and we call write() on the SSL socket, the SSL library may then need to perform a read(), which could block indefinitely. Depending on how long SSL blocks, there's no guarantee that it will return in enough time to handle the timeout properly.
        Hide
        Jakub Scholz added a comment -

        Hi Ken,

        Is there some reason why not to simply create the SSL socket as blocking? For us it definitely fixed the error which was occurring quite often during our continuous integration runs. And I do not assume it would cause any problems if the non-SSL socket is already blocking. Or do I miss something?

        Regards
        Jakub

        Show
        Jakub Scholz added a comment - Hi Ken, Is there some reason why not to simply create the SSL socket as blocking? For us it definitely fixed the error which was occurring quite often during our continuous integration runs. And I do not assume it would cause any problems if the non-SSL socket is already blocking. Or do I miss something? Regards Jakub
        Hide
        Ken Giusti added a comment -

        Root cause appears to be due to this python bug:

        http://bugs.python.org/issue8240

        Been around for awhile - probably not going to be fixed in python anytime soon.

        Need to work-around this bug.

        Show
        Ken Giusti added a comment - Root cause appears to be due to this python bug: http://bugs.python.org/issue8240 Been around for awhile - probably not going to be fixed in python anytime soon. Need to work-around this bug.
        Hide
        Ken Giusti added a comment -

        Reproducer. I have to run two or three of these in parallel before the exception is raised.

        I also run "drain -f ken > /dev/null" to keep the queue from filling.

        Show
        Ken Giusti added a comment - Reproducer. I have to run two or three of these in parallel before the exception is raised. I also run "drain -f ken > /dev/null" to keep the queue from filling.
        Hide
        Ken Giusti added a comment -
        Show
        Ken Giusti added a comment - Originally reported by Michal Zerola https://issues.apache.org/jira/secure/ViewProfile.jspa?name=zer0

          People

          • Assignee:
            Ken Giusti
            Reporter:
            Ken Giusti
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development