Qpid
  1. Qpid
  2. QPID-3642

Implement Dead Letter Queue functionality

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.15
    • Component/s: Java Broker, Java Client
    • Labels:
      None

      Description

      Implement functionality to move messages into Dead Letter Queue on reaching Max Delivery Count limit.

      The Maximum Delivery Count is a property of a queue. If the client application is unable to process a message more than a specified number of times, the then the broker either moves the message to the specified DLQ, or it is dropped (producing an operation log message).

      If the application is unable to process a message it must call either Session#recover() or Session#rollback() (depending on transactionality) on the Session in use. If during recovery the broker discovers that a message has been seen (at least) the maximum number of times then it will move the message to the DLQ or drop the message, depending on how the queue is configured.

      This feature can be enabled for a queue either through the XML configuration for the broker, or when declaring the queue using JMX. It is also necessary to enable a setting on the client. The feature is available for both synchronous consumers (i.e. those using MessageConsumer#receive()) and asynchronous consumers (i.e. those implementing MessageListener#onMessage) using all JMS delivery modes.

      When enabled for a given queue, the DLQ feature causes generation of a Dead Letter Exchange and a Dead Letter Queue, named according to the convention <queue_name>_DLE and <queue_name>_DLQ. If the client indicates it no longer wishes the receive the message (by means of the Max Delivery Count feature) then the message is routed through the dead letter exchange onto the DLQ and removed from the original queue.

        Issue Links

          Activity

          Alex Rudyy created issue -
          Alex Rudyy made changes -
          Field Original Value New Value
          Description Implement functionality to move messages into Dead Letter Queue on reaching Max Delivery Count limit. Implement functionality to move messages into Dead Letter Queue on reaching Max Delivery Count limit.

          The Maximum Delivery Count is a property of a queue. If the client application is unable to process a message more than a specified number of times, the then the broker either moves the message to the specified DLQ, or it is dropped (producing an operation log message).

          If the application is unable to process a message it must call either Session#recover() or Session#rollback() (depending on transactionality) on the Session in use. If during recovery the broker discovers that a message has been seen (at least) the maximum number of times then it will move the message to the DLQ or drop the message, depending on how the queue is configured.

          This feature can be enabled for a queue either through the XML configuration for the broker, or when declaring the queue using JMX. It is also necessary to enable a setting on the client. The feature is available for both synchronous consumers (i.e. those using MessageConsumer#receive()) and asynchronous consumers (i.e. those implementing MessageListener#onMessage) using all JMS delivery modes.

          When enabled for a given queue, the DLQ feature causes generation of a Dead Letter Exchange and a Dead Letter Queue, named according to the convention <queue_name>_DLE and <queue_name>_DLQ. If the client indicates it no longer wishes the receive the message (by means of the Max Delivery Count feature) then the message is routed through the dead letter exchange onto the DLQ and removed from the original queue.
          Alex Rudyy made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Alex Rudyy made changes -
          Attachment 0001-QPID-3642-QPID-3643-Add-Dead-Letter-Queue-functional.patch [ 12504893 ]
          Alex Rudyy made changes -
          Status In Progress [ 3 ] Ready To Review [ 10006 ]
          Hide
          Alex Rudyy added a comment -

          Keith, could you please review and commit the patch attached?

          Show
          Alex Rudyy added a comment - Keith, could you please review and commit the patch attached?
          Alex Rudyy made changes -
          Assignee Alex Rudyy [ alex.rufous ] Keith Wall [ k-wall ]
          Alex Rudyy made changes -
          Attachment 0001-QPID-3642-QPID-3643-Add-Dead-Letter-Queue-functional.patch [ 12504893 ]
          Alex Rudyy made changes -
          Attachment 0001-QPID-3642-QPID-3643-Add-Dead-Letter-Queue-functional.patch [ 12504899 ]
          Alex Rudyy made changes -
          Component/s Java Client [ 12311389 ]
          Hide
          Keith Wall added a comment -

          Hi Alex

          Couple of comments:

          1) Can you extract the separate bug fix we discussed into a separate Jira/patch?
          2) AMQBrokerManagerMBean#createNewQueue surely the old method should call the new override, and avaoid duplicating the method.
          3) AMQConnection.java could you document that rejectBehaviour member only applies to 0-8..0-9-1.

          Thanks, Keith.

          Show
          Keith Wall added a comment - Hi Alex Couple of comments: 1) Can you extract the separate bug fix we discussed into a separate Jira/patch? 2) AMQBrokerManagerMBean#createNewQueue surely the old method should call the new override, and avaoid duplicating the method. 3) AMQConnection.java could you document that rejectBehaviour member only applies to 0-8..0-9-1. Thanks, Keith.
          Keith Wall made changes -
          Assignee Keith Wall [ k-wall ] Alex Rudyy [ alex.rufous ]
          Alex Rudyy made changes -
          Attachment 0001-QPID-3642-QPID-3643-Add-Dead-Letter-Queue-functional.patch [ 12504899 ]
          Alex Rudyy made changes -
          Attachment 0001-QPID-3642-QPID-3643-Add-Dead-Letter-Queue-functional.patch [ 12505026 ]
          Alex Rudyy made changes -
          Attachment 0002-QPID-3642-QPID-3640-Add-Dead-Letter-Queue-functional.patch [ 12505027 ]
          Hide
          Alex Rudyy added a comment -

          Keith,

          > 1) Can you extract the separate bug fix we discussed into a separate Jira/patch?

          I extracted changes into 2 patches.
          The first patch adds DLQ & max delivery count functionality for 0-8/0-9/0-9-1 paths.
          The second patch adds DLQ & max delivery count functionality for 0-10 paths.
          Also, in second patch I fixed issue with setting of re-delivery flag for pre-fetched messages(QPID-3640).

          > 2) AMQBrokerManagerMBean#createNewQueue surely the old method should call the new override, and avoid duplicating the method.

          Fixed.

          > 3) AMQConnection.java could you document that rejectBehaviour member only applies to 0-8..0-9-1.

          I moved getRejectBehaviour functionality from AMQConnection into BasicMessageConsumer_0_8 as it is the place where it is needed.
          However, I could not moved AMQDestinaton#getRejectBehaviour into BasicMessageConsumer, so, I added more commentaries about "rejectbehaviour" option in AMQDestinaton.

          Show
          Alex Rudyy added a comment - Keith, > 1) Can you extract the separate bug fix we discussed into a separate Jira/patch? I extracted changes into 2 patches. The first patch adds DLQ & max delivery count functionality for 0-8/0-9/0-9-1 paths. The second patch adds DLQ & max delivery count functionality for 0-10 paths. Also, in second patch I fixed issue with setting of re-delivery flag for pre-fetched messages( QPID-3640 ). > 2) AMQBrokerManagerMBean#createNewQueue surely the old method should call the new override, and avoid duplicating the method. Fixed. > 3) AMQConnection.java could you document that rejectBehaviour member only applies to 0-8..0-9-1. I moved getRejectBehaviour functionality from AMQConnection into BasicMessageConsumer_0_8 as it is the place where it is needed. However, I could not moved AMQDestinaton#getRejectBehaviour into BasicMessageConsumer, so, I added more commentaries about "rejectbehaviour" option in AMQDestinaton.
          Alex Rudyy made changes -
          Attachment 0002-QPID-3642-QPID-3640-Add-Dead-Letter-Queue-functional.patch [ 12505027 ]
          Alex Rudyy made changes -
          Attachment 0001-QPID-3642-QPID-3643-Add-Dead-Letter-Queue-functional.patch [ 12505026 ]
          Alex Rudyy made changes -
          Attachment 0001-QPID-3642-QPID-3643-Add-Dead-Letter-Queue-functional.patch [ 12505108 ]
          Attachment 0002-QPID-3642-QPID-3640-Add-Dead-Letter-Queue-functional.patch [ 12505109 ]
          Hide
          Alex Rudyy added a comment -

          Updated patches to fix JMX API version and docs.
          Refactored client to store pre-fetched message tags in a separate queue.

          Show
          Alex Rudyy added a comment - Updated patches to fix JMX API version and docs. Refactored client to store pre-fetched message tags in a separate queue.
          Hide
          Keith Wall added a comment - - edited

          Hi Alex/Andy

          My only comment is that AMQBrokerManagerMBean#createNewQueue has been changed to chain tyhe AMQException to the JMException. It is likely this will cause a unmarshall exception on the client as the client wont have the class. No other comments.

          cheers Keith

          Show
          Keith Wall added a comment - - edited Hi Alex/Andy My only comment is that AMQBrokerManagerMBean#createNewQueue has been changed to chain tyhe AMQException to the JMException. It is likely this will cause a unmarshall exception on the client as the client wont have the class. No other comments. cheers Keith
          Alex Rudyy made changes -
          Attachment 0001-QPID-3642-QPID-3643-Add-Dead-Letter-Queue-functional.patch [ 12505108 ]
          Alex Rudyy made changes -
          Hide
          Alex Rudyy added a comment -

          Keith,

          I changed AMQBrokerManagerMBean#createNewQueue and updated patch.

          Show
          Alex Rudyy added a comment - Keith, I changed AMQBrokerManagerMBean#createNewQueue and updated patch.
          Hide
          Keith Wall added a comment -

          Patches applied.

          Show
          Keith Wall added a comment - Patches applied.
          Keith Wall made changes -
          Status Ready To Review [ 10006 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Keith Wall added a comment -

          As discussed, causing a regression failure against the Python test suite:

          https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Python-Java-Test/35/

          • qpid.tests.messaging.endpoints.ReceiverTests.testAtMostOnce
          • qpid.tests.messaging.endpoints.ReceiverTests.testUnreliable
          Show
          Keith Wall added a comment - As discussed, causing a regression failure against the Python test suite: https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Python-Java-Test/35/ qpid.tests.messaging.endpoints.ReceiverTests.testAtMostOnce qpid.tests.messaging.endpoints.ReceiverTests.testUnreliable
          Keith Wall made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          Andrew MacBean added a comment -

          Regression caused by our use of the redelivered flag when in ServerSession.onClose we call release on the subscriptions. Fixed this and changed the logic in subscription release method to handle correctly. Also added equivalent Java test for this case.

          Subsequent patch attached.

          Show
          Andrew MacBean added a comment - Regression caused by our use of the redelivered flag when in ServerSession.onClose we call release on the subscriptions. Fixed this and changed the logic in subscription release method to handle correctly. Also added equivalent Java test for this case. Subsequent patch attached.
          Andrew MacBean made changes -
          Andrew MacBean made changes -
          Assignee Alex Rudyy [ alex.rufous ] Andrew MacBean [ macbean ]
          Andrew MacBean made changes -
          Status Reopened [ 4 ] In Progress [ 3 ]
          Andrew MacBean made changes -
          Status In Progress [ 3 ] Ready To Review [ 10006 ]
          Andrew MacBean made changes -
          Assignee Andrew MacBean [ macbean ] Keith Wall [ k-wall ]
          Hide
          Keith Wall added a comment -

          Patch applied.

          Show
          Keith Wall added a comment - Patch applied.
          Keith Wall made changes -
          Status Ready To Review [ 10006 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Keith Wall made changes -
          Link This issue is depended upon by QPID-4502 [ QPID-4502 ]
          Rob Godfrey made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          39m 5s 1 Alex Rudyy 23/Nov/11 17:20
          Resolved Resolved Reopened Reopened
          1d 1h 44m 1 Keith Wall 29/Nov/11 11:04
          Reopened Reopened In Progress In Progress
          23h 51m 1 Andrew MacBean 30/Nov/11 10:56
          In Progress In Progress Reviewable Reviewable
          1m 26s 2 Andrew MacBean 30/Nov/11 10:56
          Reviewable Reviewable Resolved Resolved
          4d 18h 59m 2 Keith Wall 30/Nov/11 13:58
          Resolved Resolved Closed Closed
          1169d 6h 9m 1 Rob Godfrey 11/Feb/15 20:07

            People

            • Assignee:
              Keith Wall
              Reporter:
              Alex Rudyy
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development