Uploaded image for project: 'ActiveMQ'
  1. ActiveMQ
  2. AMQ-5187

Virtual destination consumers do not support retroactive message recovery

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.9.1
    • Fix Version/s: 5.11.0
    • Component/s: Broker
    • Labels:
      None

      Description

      Virtual destinations intercept messages sent by producers and pass them onto consumers, but when consumers connect to mapped destinations, they do not receive retroactive messages.
      This is particularly required for MQTT clients using virtual topics to send retained messages to consumer queues.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user dhirajsb opened a pull request:

          https://github.com/apache/activemq/pull/24

          Fixed AMQ-5187 added support for virtual destination subscription recovery

          This patch adds support for applying subscription recovery policy to mapped Queues for virtual destinations. This is a MUST for mapping MQTT retained messages from Virtual/Composite Topics to mapped Queues.

          Since it depends on AMQ-5160, it Includes commits from https://github.com/apache/activemq/pull/22.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/dhirajsb/activemq AMQ-5187

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/activemq/pull/24.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #24


          commit d45de03ec781a37deaf959202f6549f076be4c2a
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-08T02:05:36Z

          Partial fix for AMQ-5160, attempts to resolve retained messages using subscription recovery policy, but fails to resend retained messages for duplicate subscriptions

          commit e510d129eb80f493d300e2f5fe370c83f1d946ad
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-12T19:37:54Z

          Fixed AMQ-5160, Added RegionBroker.getConnectionContext(), made Topic.recoverRetroactiveMessages() public to force message recovery from MQTTProtocolConverter for duplicate subscriptions, added new tests for retained messages and JMX interoperability

          commit 33e663da38e737aa13200dbf5c10a231465fe0e6
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-13T02:51:21Z

          Fixed AMQ-5160, fixed browse() to include messages from wrapped policy

          commit 5648cd37febf9363397fc2b97c5f23c45427e9fc
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-13T03:25:11Z

          Fixed AMQ-5160, removed redundant MQTTRetainedMessages

          commit b579dc472e340a15e45675df2c70a133fcbf1b61
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-13T07:29:03Z

          Fixed AMQ-5160, force durable subscriptions to always recover retroactive messages

          commit 7fe6dc0efa9cf645210758907d158f1874a3b5bf
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-13T07:29:40Z

          Fixed AMQ-5160, remove durable subscription in onUnsubscribe()

          commit 4e611393cd857ae08bb2e7412631b560af52350c
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-13T07:30:11Z

          Fixed AMQ-5160, polished MQTT tests

          commit 0ea3513759caab52083e402d558d6c864661dfad
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-13T07:45:05Z

          Fixed AMQ-5160, fixed test testNoMessageReceivedAfterUnsubscribeMQTT

          commit b29112e49219c8ce281920687d1244d7bb3b5074
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-13T19:14:21Z

          Fixed AMQ-5160, changed DurableTopicSubscription to only recover retroactive messages for retroactive topics or consumers

          commit fb3008f6644c834bbd54c917729ed246fb96fc6a
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-13T19:15:14Z

          Fixed AMQ-5160, removed producer's retain property from retained messages

          commit a0ee487b0dd9ac17ae89fe07c365cb5a2603e9c1
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-13T19:16:44Z

          Fixed AMQ-5160, fixed race condition for retained messages

          commit 30d23491ac900d6e8ac4f0fd64ea86e2e3f0fe7b
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-13T20:15:04Z

          Fixed AMQ-5160, restored previous DurableSubscription behaviour of only recovering messages when cursor is empty, retained messages are always recovered

          commit 01749352991a2b0bceaec8896d6a3493ea15487d
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-16T21:21:19Z

          Fixed AMQ-5160, fixed durable subscription retroactive recovery

          commit f0c3b9675e216221ba6d8a2aa9d23620f7206a78
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-17T03:03:50Z

          AMQ-5187, updated virtual destinations to support recovering retractive messages from virtual and composite topics, added MQTT tests to verify retain message recovery

          commit 5e8a763d8f8493b858d2a20cab98defbc53ec935
          Author: Dhiraj Bokde <dhirajsb@yahoo.com>
          Date: 2014-05-17T03:17:26Z

          AMQ-5187, updated test to verify that second virtual topic queue consumer does not receive retained message


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user dhirajsb opened a pull request: https://github.com/apache/activemq/pull/24 Fixed AMQ-5187 added support for virtual destination subscription recovery This patch adds support for applying subscription recovery policy to mapped Queues for virtual destinations. This is a MUST for mapping MQTT retained messages from Virtual/Composite Topics to mapped Queues. Since it depends on AMQ-5160 , it Includes commits from https://github.com/apache/activemq/pull/22 . You can merge this pull request into a Git repository by running: $ git pull https://github.com/dhirajsb/activemq AMQ-5187 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq/pull/24.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #24 commit d45de03ec781a37deaf959202f6549f076be4c2a Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-08T02:05:36Z Partial fix for AMQ-5160 , attempts to resolve retained messages using subscription recovery policy, but fails to resend retained messages for duplicate subscriptions commit e510d129eb80f493d300e2f5fe370c83f1d946ad Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-12T19:37:54Z Fixed AMQ-5160 , Added RegionBroker.getConnectionContext(), made Topic.recoverRetroactiveMessages() public to force message recovery from MQTTProtocolConverter for duplicate subscriptions, added new tests for retained messages and JMX interoperability commit 33e663da38e737aa13200dbf5c10a231465fe0e6 Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-13T02:51:21Z Fixed AMQ-5160 , fixed browse() to include messages from wrapped policy commit 5648cd37febf9363397fc2b97c5f23c45427e9fc Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-13T03:25:11Z Fixed AMQ-5160 , removed redundant MQTTRetainedMessages commit b579dc472e340a15e45675df2c70a133fcbf1b61 Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-13T07:29:03Z Fixed AMQ-5160 , force durable subscriptions to always recover retroactive messages commit 7fe6dc0efa9cf645210758907d158f1874a3b5bf Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-13T07:29:40Z Fixed AMQ-5160 , remove durable subscription in onUnsubscribe() commit 4e611393cd857ae08bb2e7412631b560af52350c Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-13T07:30:11Z Fixed AMQ-5160 , polished MQTT tests commit 0ea3513759caab52083e402d558d6c864661dfad Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-13T07:45:05Z Fixed AMQ-5160 , fixed test testNoMessageReceivedAfterUnsubscribeMQTT commit b29112e49219c8ce281920687d1244d7bb3b5074 Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-13T19:14:21Z Fixed AMQ-5160 , changed DurableTopicSubscription to only recover retroactive messages for retroactive topics or consumers commit fb3008f6644c834bbd54c917729ed246fb96fc6a Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-13T19:15:14Z Fixed AMQ-5160 , removed producer's retain property from retained messages commit a0ee487b0dd9ac17ae89fe07c365cb5a2603e9c1 Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-13T19:16:44Z Fixed AMQ-5160 , fixed race condition for retained messages commit 30d23491ac900d6e8ac4f0fd64ea86e2e3f0fe7b Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-13T20:15:04Z Fixed AMQ-5160 , restored previous DurableSubscription behaviour of only recovering messages when cursor is empty, retained messages are always recovered commit 01749352991a2b0bceaec8896d6a3493ea15487d Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-16T21:21:19Z Fixed AMQ-5160 , fixed durable subscription retroactive recovery commit f0c3b9675e216221ba6d8a2aa9d23620f7206a78 Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-17T03:03:50Z AMQ-5187 , updated virtual destinations to support recovering retractive messages from virtual and composite topics, added MQTT tests to verify retain message recovery commit 5e8a763d8f8493b858d2a20cab98defbc53ec935 Author: Dhiraj Bokde <dhirajsb@yahoo.com> Date: 2014-05-17T03:17:26Z AMQ-5187 , updated test to verify that second virtual topic queue consumer does not receive retained message
          Hide
          dhirajsb Dhiraj Bokde added a comment -
          Show
          dhirajsb Dhiraj Bokde added a comment - I've also written a post about this fix at http://rockablogbaby.blogspot.com/2014/05/scalable-iot-integration-using-apache.html
          Hide
          tabish121 Timothy Bish added a comment -

          It looks like your PR-22 was merged which contained some of the work in PR-24. Problem is that given other changes that have gone in 24 won't apply cleanly. Can you take a look and perhaps rebase against trunk to produce an updated pull request that will work with current trunk code?

          Show
          tabish121 Timothy Bish added a comment - It looks like your PR-22 was merged which contained some of the work in PR-24. Problem is that given other changes that have gone in 24 won't apply cleanly. Can you take a look and perhaps rebase against trunk to produce an updated pull request that will work with current trunk code?
          Hide
          dhirajsb Dhiraj Bokde added a comment -

          I've resolved conflicts and updated my patch branch. Please try it again.

          Show
          dhirajsb Dhiraj Bokde added a comment - I've resolved conflicts and updated my patch branch. Please try it again.
          Hide
          tabish121 Timothy Bish added a comment -

          The patch now has commits for issues already applied on trunk so it still produces a lot of conflicts. Maybe just merge you branch with trunk and then create a patch by comparison via: git format-patch trunk >> mqtt.patch

          Show
          tabish121 Timothy Bish added a comment - The patch now has commits for issues already applied on trunk so it still produces a lot of conflicts. Maybe just merge you branch with trunk and then create a patch by comparison via: git format-patch trunk >> mqtt.patch
          Hide
          dhirajsb Dhiraj Bokde added a comment -

          Sorry about the branch mess. I am attaching the result of

          git format-patch trunk >> AMQ-5187.patch
          
          Show
          dhirajsb Dhiraj Bokde added a comment - Sorry about the branch mess. I am attaching the result of git format-patch trunk >> AMQ-5187.patch
          Hide
          tabish121 Timothy Bish added a comment -

          This doesn't seem to apply entirely cleanly to trunk either, bits get dropped due to conflicts.

          Show
          tabish121 Timothy Bish added a comment - This doesn't seem to apply entirely cleanly to trunk either, bits get dropped due to conflicts.
          Hide
          dhirajsb Dhiraj Bokde added a comment -

          Merged master repo trunk branch into AMQ-5187 branch. Should be able to apply the github PR now.

          Show
          dhirajsb Dhiraj Bokde added a comment - Merged master repo trunk branch into AMQ-5187 branch. Should be able to apply the github PR now.
          Hide
          tabish121 Timothy Bish added a comment -

          Applied patch with a few small tweaks. Moved the composite test case to it's own test since it requires specific Broker config and doesn't really need to be tested on each transport type.

          Show
          tabish121 Timothy Bish added a comment - Applied patch with a few small tweaks. Moved the composite test case to it's own test since it requires specific Broker config and doesn't really need to be tested on each transport type.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/activemq/pull/24

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/activemq/pull/24

            People

            • Assignee:
              tabish121 Timothy Bish
              Reporter:
              dhirajsb Dhiraj Bokde
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development