[QPID-3767] Federation link index becomes invalid on failover against a cluster.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.14
• Fix Version/s: 0.17
• Component/s: C++ Broker
• Assignee: Ken Giusti
• Reporter: Ken Giusti
• Created: 17/Jan/12 20:39
• Updated: 29/Jul/13 18:53
• Resolved: 09/Aug/12 13:41

Description

The Link management object that represents a connection between two federated brokers is indexed (identified) by the remote broker's host and port. If the remote broker is part of a cluster, and a failover event occurs, the host:port used by the Link object's index may no longer exist. This prevents the route from being deleted.

For example, create a cluster of two brokers using addresses 127.0.0.1:2222 and 127.0.0.1:3333. Start a third broker, say 127.0.0.1:8888. Create a queue route from 127.0.0.1:2222 to an exchange on 127.0.0.1:8888. Kill the broker 127.0.0.1:2222. This results in a Link object that is connected to 127.0.0.1:3333, but reports 127.0.0.1:2222 as it's host.

[kgiusti@localhost src]$ qpid-config -a 127.0.0.1:2222 add queue src
[kgiusti@localhost src]$ qpid-config -a 127.0.0.1:8888 add exchange fanout destx
[kgiusti@localhost src]$ qpid-config -a 127.0.0.1:8888 add queue dest
[kgiusti@localhost src]$ qpid-config -a 127.0.0.1:8888 bind destx dest
[kgiusti@localhost src]$ qpid-route queue add 127.0.0.1:8888 127.0.0.1:2222 destx src
[kgiusti@localhost src]$ ../examples/messaging/spout -b 127.0.0.1:2222 --content "ZZZ" src
[kgiusti@localhost src]$ ../examples/messaging/drain -b 127.0.0.1:8888 -t 2 dest
Message(properties=

{spout-id:8c308c74-6b25-4408-8694-93ef8352a308:0, x-amqp-0-10.routing-key:src}

, content='ZZZ')

<Kill Broker 127.0.0.1:2222, link fails over to 127.0.0.1:3333>

From qpid-tool:

qpid: show 133
Object of type: org.apache.qpid.broker:link:_data(bc33c1b3-25cd-e0ce-04d7-ad684ed36d91)
Attribute 133
=================================================
vhostRef 150
host 127.0.0.1
port 2222
transport tcp
durable False
state Operational
lastError Failed over to tcp:10.16.185.15:3333

Once this occurs, I am unable to delete the link:

[kgiusti@localhost src]$ qpid-route queue del 127.0.0.1:8888 127.0.0.1:2222 destx src
[kgiusti@localhost src]$ qpid-tool 127.0.0.1:8888

qpid: list
Summary of Objects by Type:
Package Class Active Deleted
=======================================================
org.apache.qpid.broker binding 14 0
org.apache.qpid.broker system 1 0
org.apache.qpid.broker broker 1 0
org.apache.qpid.broker bridge 1 0
org.apache.qpid.broker link 1 0
org.apache.qpid.broker subscription 5 0
org.apache.qpid.broker connection 2 0
org.apache.qpid.broker session 2 0
org.apache.qpid.broker queue 6 0
org.apache.qpid.broker exchange 9 0
org.apache.qpid.broker vhost 1 0

Issue Links

• breaks: QPID-4347 durable links break recovery