Qpid
  1. Qpid
  2. QPID-3810

[Java broker] stop the acceptor thread exiting when encountering IOExceptions such as too many open files

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.14
    • Fix Version/s: 0.15
    • Component/s: Java Broker
    • Labels:
      None

      Description

      The Java brokers new IO acceptor thread will exit when encountering IOExceptions such as too many open files. It should instead catch the Exception and attempt to continue (after a short delay to prevent tight spinning throwing such exceptions) and allow broker operation to continue if the issue is resolved.

        Activity

        Robbie Gemmell created issue -
        Robbie Gemmell made changes -
        Field Original Value New Value
        Status Open [ 1 ] In Progress [ 3 ]
        Robbie Gemmell made changes -
        Status In Progress [ 3 ] Ready To Review [ 10006 ]
        Hide
        Robbie Gemmell added a comment -

        Keith, could you review this change please? Thanks, Robbie.

        Show
        Robbie Gemmell added a comment - Keith, could you review this change please? Thanks, Robbie.
        Robbie Gemmell made changes -
        Assignee Robbie Gemmell [ gemmellr ] Keith Wall [ k-wall ]
        Hide
        Keith Wall added a comment -

        Hi Robbie,

        A couple of comments, most of which could be equally applied to the previous version:

        1) In the InterruptedException catch block, we should either re-interrupt ourselves, or set _closed = true, to make sure that the AcceptingThread responds correctly to an interrupt and shuts itself down.

        2) In the event of a RuntimeException (line 223) or IOException (line 227), we should close any (non-null) Socket that was accepted (line 206) to avoid a potential socket leak and ensure that the peer gets a exception.

        3) Guard the new debug in the finally with isDebugEnabled.

        HTH, Keith.

        Show
        Keith Wall added a comment - Hi Robbie, A couple of comments, most of which could be equally applied to the previous version: 1) In the InterruptedException catch block, we should either re-interrupt ourselves, or set _closed = true, to make sure that the AcceptingThread responds correctly to an interrupt and shuts itself down. 2) In the event of a RuntimeException (line 223) or IOException (line 227), we should close any (non-null) Socket that was accepted (line 206) to avoid a potential socket leak and ensure that the peer gets a exception. 3) Guard the new debug in the finally with isDebugEnabled. HTH, Keith.
        Keith Wall made changes -
        Assignee Keith Wall [ k-wall ] Robbie Gemmell [ gemmellr ]
        Hide
        Robbie Gemmell added a comment -

        Changes made, can you look it over again Keith?

        Thanks,
        Robbie

        Show
        Robbie Gemmell added a comment - Changes made, can you look it over again Keith? Thanks, Robbie
        Robbie Gemmell made changes -
        Assignee Robbie Gemmell [ gemmellr ] Keith Wall [ k-wall ]
        Hide
        Keith Wall added a comment -

        Changes look fine, but I've just noticed the switch in logger seems to be meaning we are seeing format patterns:

        main 2012-02-04 19:39:12,552 DEBUG [apache.qpid.client.AMQConnectionDelegate_0_10] connecting to host: localhost port: 15672 vhost: test username: guest password: ********
        main 2012-02-04 19:39:12,616 DEBUG [transport.network.io.IoNetworkTransport] SO_RCVBUF : %s
        main 2012-02-04 19:39:12,616 DEBUG [transport.network.io.IoNetworkTransport] SO_SNDBUF : %s
        main 2012-02-04 19:39:12,616 DEBUG [transport.network.io.IoNetworkTransport] TCP_NODELAY : %s
        main 2012-02-04 19:39:12,646 DEBUG [apache.qpid.transport.Connection] SEND: [conn:26796e1c] AMQP.1 0-10
        
        Show
        Keith Wall added a comment - Changes look fine, but I've just noticed the switch in logger seems to be meaning we are seeing format patterns: main 2012-02-04 19:39:12,552 DEBUG [apache.qpid.client.AMQConnectionDelegate_0_10] connecting to host: localhost port: 15672 vhost: test username: guest password: ******** main 2012-02-04 19:39:12,616 DEBUG [transport.network.io.IoNetworkTransport] SO_RCVBUF : %s main 2012-02-04 19:39:12,616 DEBUG [transport.network.io.IoNetworkTransport] SO_SNDBUF : %s main 2012-02-04 19:39:12,616 DEBUG [transport.network.io.IoNetworkTransport] TCP_NODELAY : %s main 2012-02-04 19:39:12,646 DEBUG [apache.qpid.transport.Connection] SEND: [conn:26796e1c] AMQP.1 0-10
        Keith Wall made changes -
        Assignee Keith Wall [ k-wall ] Robbie Gemmell [ gemmellr ]
        Hide
        Robbie Gemmell added a comment -

        Oops, didn't notice those. Updated now, 3rd time lucky?

        Show
        Robbie Gemmell added a comment - Oops, didn't notice those. Updated now, 3rd time lucky?
        Hide
        Keith Wall added a comment -

        Looks good to me.

        Show
        Keith Wall added a comment - Looks good to me.
        Keith Wall made changes -
        Status Ready To Review [ 10006 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Rob Godfrey made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Robbie Gemmell
            Reporter:
            Robbie Gemmell
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development