ActiveMQ
  1. ActiveMQ
  2. AMQ-3124

Failover transport client gets corrupted connectedBrokers data

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 5.4.2
    • Fix Version/s: 5.5.0
    • Component/s: Broker
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      When clients use the Failover transport, the broker will deliver a list of its own uri and also brokers it peers with to the client. This happens on initial connection and, if configured, as brokers come in and out of the cluster.

      The problem is that the entire URI is given, including query string information. If the query string contains commas in it then the Failover client breaks it up into unusable client uris and will attempt to connect to them. Also, in 5.4.2, this leads to the same failure that occurs in AMQ-3085.

      Also, there doesn't seem to be a point in the client trying to use the same query string parameters that the server is using.

      Attached patch strips the query string before sending uris to the client.

      Example:

      Broker1:

      <transportConnectors>
      <transportConnector name="openwire+ssl" uri="ssl://0.0.0.0:61616?needClientAuth=true&enabledCipherSuites=SSL_RSA_WITH_RC4_128_SHA,SSL_RSA_WITH_3DES_EDE_CBC_SHA,SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA,SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_DSS_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA" />
      </transportConnectors>

      With a client connecting to: failover:(ssl://broker1:61616), you will see (extra debugging added by me):

      [DEBUG] FailoverTransport - Failover add: u[0] is: ssl://broker1:61616
      [DEBUG] FailoverTransport - Reconnect was triggered but transport is not started yet. Wait for start to connect the transport.
      [DEBUG] FailoverTransport - Started.
      [DEBUG] FailoverTransport - Waking up reconnect task
      [DEBUG] FailoverTransport - urlList connectionList:[ssl://broker1:61616]
      [DEBUG] FailoverTransport - Attempting connect to: ssl://broker1:61616
      [DEBUG] FailoverTransport - Failover add: u[0] is: TLS_RSA_WITH_AES_256_CBC_SHA
      [DEBUG] FailoverTransport - Failover add: u[1] is: TLS_DHE_DSS_WITH_AES_256_CBC_SHA
      [DEBUG] FailoverTransport - Failover add: u[2] is: TLS_RSA_WITH_AES_128_CBC_SHA
      [DEBUG] FailoverTransport - Failover add: u[3] is: TLS_DHE_RSA_WITH_AES_128_CBC_SHA
      [DEBUG] FailoverTransport - Failover add: u[4] is: ssl://broker1:61616?needClientAuth=true&enabledCipherSuites=SSL_RSA_WITH_RC4_128_SHA
      [DEBUG] FailoverTransport - Failover add: u[5] is: TLS_DHE_DSS_WITH_AES_128_CBC_SHA
      [WARN] FailoverTransport - Transport (broker1/10.73.76.102:61616) failed to ssl://broker1:61616 , attempting to automatically reconnect due to: java.io.IOException: Unexpected error occured
      [DEBUG] FailoverTransport - Transport failed with the following exception: <java.io.IOException: Unexpected error occured>java.io.IOException: Unexpected error occured
      at org.apache.activemq.transport.failover.FailoverTransport.add(FailoverTransport.java:614)
      at org.apache.activemq.transport.failover.FailoverTransport.updateURIs(FailoverTransport.java:1049)
      at org.apache.activemq.transport.failover.FailoverTransport.processNewTransports(FailoverTransport.java:285)
      at org.apache.activemq.transport.failover.FailoverTransport.handleConnectionControl(FailoverTransport.java:265)
      at org.apache.activemq.transport.failover.FailoverTransport$3.onCommand(FailoverTransport.java:177)
      [DEBUG] FailoverTransport - urlList connectionList:[TLS_RSA_WITH_AES_256_CBC_SHA, ssl://broker1:61616]
      [DEBUG] FailoverTransport - Attempting connect to: TLS_RSA_WITH_AES_256_CBC_SHA
      [DEBUG] FailoverTransport - Connect fail to: TLS_RSA_WITH_AES_256_CBC_SHA, reason: java.io.IOException: Transport not scheme specified: [TLS_RSA_WITH_AES_256_CBC_SHA]
      [DEBUG] FailoverTransport - Attempting connect to: ssl://broker1:61616
      [DEBUG] FailoverTransport - Connection established
      [INFO] FailoverTransport - Successfully reconnected to ssl://broker1:61616

      Note that the stacktrace occurs for the same reason as AMQ-3085, but all the extra work attempting to connect to invalid uris is wasteful. Also, the client has no need for the query string parameters of the server uris.

      1. amq-3124.patch
        1 kB
        Adam Sussman

        Issue Links

          Activity

          Adam Sussman created issue -
          Hide
          Adam Sussman added a comment -

          Proposed patch. Strips query string from uris.

          Show
          Adam Sussman added a comment - Proposed patch. Strips query string from uris.
          Adam Sussman made changes -
          Field Original Value New Value
          Attachment amq-3124.patch [ 12467784 ]
          Adam Sussman made changes -
          Description When clients use the Failover transport, the broker will deliver a list of brokers it peers with to the client. This happens on initial connection and, if configured, as brokers come in and out of the cluster.

          The problem is that the entire URI is given, including query string information. If the query string contains commas in it then the Failover client breaks it up into unusable client uris and will attempt to connect to them. Also, in 5.4.2, this leads to the same failure that occurs in AMQ-3085.

          Also, there doesn't seem to be a point in the client trying to use the same query string parameters that the server is using.

          Attached patch strips the query string before sending uris to the client.

          Example:

          Broker1:

                  <transportConnectors>
                      <transportConnector name="openwire+ssl" uri="ssl://0.0.0.0:61616?needClientAuth=true&amp;enabledCipherSuites=SSL_RSA_WITH_RC4_128_SHA,SSL_RSA_WITH_3DES_EDE_CBC_SHA,SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA,SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_DSS_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA" />
                  </transportConnectors>

          Broker2:
                  <networkConnectors>
                       <networkConnector name="core1" uri="static:(ssl://broker1:61616)" duplex="true" />
                  </networkConnectors>


                  <transportConnectors>
                      <transportConnector name="openwire+ssl" uri="ssl://0.0.0.0:61616?needClientAuth=true" />
                  </transportConnectors>

          With a client connecting to: failover:(ssl://broker2:61616), you will see (extra debugging added by me):

          [DEBUG] FailoverTransport - Failover add: u[0] is: ssl://broker2:61616
          [DEBUG] FailoverTransport - Reconnect was triggered but transport is not started yet. Wait for start to connect the transport.
          [DEBUG] FailoverTransport - Started.
          [DEBUG] FailoverTransport - Waking up reconnect task
          [DEBUG] FailoverTransport - urlList connectionList:[ssl://broker2:61616]
          [DEBUG] FailoverTransport - Attempting connect to: ssl://broker2:61616
          [DEBUG] FailoverTransport - Failover add: u[0] is: TLS_RSA_WITH_AES_256_CBC_SHA
          [DEBUG] FailoverTransport - Failover add: u[1] is: TLS_DHE_DSS_WITH_AES_256_CBC_SHA
          [DEBUG] FailoverTransport - Failover add: u[2] is: TLS_RSA_WITH_AES_128_CBC_SHA
          [DEBUG] FailoverTransport - Failover add: u[3] is: TLS_DHE_RSA_WITH_AES_128_CBC_SHA
          [DEBUG] FailoverTransport - Failover add: u[4] is: ssl://broker2:61616?needClientAuth=true&enabledCipherSuites=SSL_RSA_WITH_RC4_128_SHA
          [DEBUG] FailoverTransport - Failover add: u[5] is: TLS_DHE_DSS_WITH_AES_128_CBC_SHA
          [DEBUG] FailoverTransport - Failover add: u[6] is: ssl://broker1:61616?needClientAuth=true
          [WARN] FailoverTransport - Transport (broker2/10.73.76.102:61616) failed to ssl://broker2:61616 , attempting to automatically reconnect due to: java.io.IOException: Unexpected error occured
          [DEBUG] FailoverTransport - Transport failed with the following exception: <java.io.IOException: Unexpected error occured>java.io.IOException: Unexpected error occured
          at org.apache.activemq.transport.failover.FailoverTransport.add(FailoverTransport.java:614)
          at org.apache.activemq.transport.failover.FailoverTransport.updateURIs(FailoverTransport.java:1049)
          at org.apache.activemq.transport.failover.FailoverTransport.processNewTransports(FailoverTransport.java:285)
          at org.apache.activemq.transport.failover.FailoverTransport.handleConnectionControl(FailoverTransport.java:265)
          at org.apache.activemq.transport.failover.FailoverTransport$3.onCommand(FailoverTransport.java:177)
          [DEBUG] FailoverTransport - urlList connectionList:[TLS_RSA_WITH_AES_256_CBC_SHA, ssl://broker2:61616]
          [DEBUG] FailoverTransport - Attempting connect to: TLS_RSA_WITH_AES_256_CBC_SHA
          [DEBUG] FailoverTransport - Connect fail to: TLS_RSA_WITH_AES_256_CBC_SHA, reason: java.io.IOException: Transport not scheme specified: [TLS_RSA_WITH_AES_256_CBC_SHA]
          [DEBUG] FailoverTransport - Attempting connect to: ssl://broker2:61616
          [DEBUG] FailoverTransport - Connection established
          [INFO] FailoverTransport - Successfully reconnected to ssl://broker2:61616

          Note that the stacktrace occurs for the same reason as AMQ-3085, but all the extra work attempting to connect to invalid uris is wasteful. Also, the client has no need for the query string parameters of the server uris.
          When clients use the Failover transport, the broker will deliver a list of its own uri and also brokers it peers with to the client. This happens on initial connection and, if configured, as brokers come in and out of the cluster.

          The problem is that the entire URI is given, including query string information. If the query string contains commas in it then the Failover client breaks it up into unusable client uris and will attempt to connect to them. Also, in 5.4.2, this leads to the same failure that occurs in AMQ-3085.

          Also, there doesn't seem to be a point in the client trying to use the same query string parameters that the server is using.

          Attached patch strips the query string before sending uris to the client.

          Example:

          Broker1:

                  <transportConnectors>
                      <transportConnector name="openwire+ssl" uri="ssl://0.0.0.0:61616?needClientAuth=true&amp;enabledCipherSuites=SSL_RSA_WITH_RC4_128_SHA,SSL_RSA_WITH_3DES_EDE_CBC_SHA,SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA,SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_DSS_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA" />
                  </transportConnectors>


          With a client connecting to: failover:(ssl://broker1:61616), you will see (extra debugging added by me):

          [DEBUG] FailoverTransport - Failover add: u[0] is: ssl://broker1:61616
          [DEBUG] FailoverTransport - Reconnect was triggered but transport is not started yet. Wait for start to connect the transport.
          [DEBUG] FailoverTransport - Started.
          [DEBUG] FailoverTransport - Waking up reconnect task
          [DEBUG] FailoverTransport - urlList connectionList:[ssl://broker1:61616]
          [DEBUG] FailoverTransport - Attempting connect to: ssl://broker1:61616
          [DEBUG] FailoverTransport - Failover add: u[0] is: TLS_RSA_WITH_AES_256_CBC_SHA
          [DEBUG] FailoverTransport - Failover add: u[1] is: TLS_DHE_DSS_WITH_AES_256_CBC_SHA
          [DEBUG] FailoverTransport - Failover add: u[2] is: TLS_RSA_WITH_AES_128_CBC_SHA
          [DEBUG] FailoverTransport - Failover add: u[3] is: TLS_DHE_RSA_WITH_AES_128_CBC_SHA
          [DEBUG] FailoverTransport - Failover add: u[4] is: ssl://broker1:61616?needClientAuth=true&enabledCipherSuites=SSL_RSA_WITH_RC4_128_SHA
          [DEBUG] FailoverTransport - Failover add: u[5] is: TLS_DHE_DSS_WITH_AES_128_CBC_SHA
          [WARN] FailoverTransport - Transport (broker1/10.73.76.102:61616) failed to ssl://broker1:61616 , attempting to automatically reconnect due to: java.io.IOException: Unexpected error occured
          [DEBUG] FailoverTransport - Transport failed with the following exception: <java.io.IOException: Unexpected error occured>java.io.IOException: Unexpected error occured
          at org.apache.activemq.transport.failover.FailoverTransport.add(FailoverTransport.java:614)
          at org.apache.activemq.transport.failover.FailoverTransport.updateURIs(FailoverTransport.java:1049)
          at org.apache.activemq.transport.failover.FailoverTransport.processNewTransports(FailoverTransport.java:285)
          at org.apache.activemq.transport.failover.FailoverTransport.handleConnectionControl(FailoverTransport.java:265)
          at org.apache.activemq.transport.failover.FailoverTransport$3.onCommand(FailoverTransport.java:177)
          [DEBUG] FailoverTransport - urlList connectionList:[TLS_RSA_WITH_AES_256_CBC_SHA, ssl://broker1:61616]
          [DEBUG] FailoverTransport - Attempting connect to: TLS_RSA_WITH_AES_256_CBC_SHA
          [DEBUG] FailoverTransport - Connect fail to: TLS_RSA_WITH_AES_256_CBC_SHA, reason: java.io.IOException: Transport not scheme specified: [TLS_RSA_WITH_AES_256_CBC_SHA]
          [DEBUG] FailoverTransport - Attempting connect to: ssl://broker1:61616
          [DEBUG] FailoverTransport - Connection established
          [INFO] FailoverTransport - Successfully reconnected to ssl://broker1:61616

          Note that the stacktrace occurs for the same reason as AMQ-3085, but all the extra work attempting to connect to invalid uris is wasteful. Also, the client has no need for the query string parameters of the server uris.
          Gary Tully made changes -
          Assignee Gary Tully [ gtully ]
          Hide
          Gary Tully added a comment -

          fix applied in r1057565
          reused TransportConnetor.getPublishableConnectString() for the propagated uri in cluster update

          Show
          Gary Tully added a comment - fix applied in r1057565 reused TransportConnetor.getPublishableConnectString() for the propagated uri in cluster update
          Gary Tully made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 5.5.0 [ 12315626 ]
          Resolution Fixed [ 1 ]
          Gary Tully made changes -
          Link This issue relates to AMQ-2632 [ AMQ-2632 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          3d 8h 31m 1 Gary Tully 11/Jan/11 10:30

            People

            • Assignee:
              Gary Tully
              Reporter:
              Adam Sussman
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development