Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-4575

Persistant Dead Letter Channel to avoid memory consumption between retries

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: 3.0.0, Future
    • Component/s: camel-core
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      When using a DeadLetterChannel, the messages are stored in memory between the retries (redeliveries). They are flushed from the memory only when we get the maximumRedeliveries value.

      It means that we can have an important memory consumption, because the messages are memory resident for a long time when:

      • if we have an important maximumRedeliveries, especially if we have -1
      • if we have an important redeliveryDelay

      I propose to create a PersistentDeadLetterChannel, working like the DeadLetterChannel, but, between redeliveries, the messages are flushed to a persistent store (filesystem, JMQ queue, JDBC, ...).

        Activity

        Hide
        davsclaus Claus Ibsen added a comment -

        There is some logic in DefaultExchangeHolder to serialize/unserialize a Camel Exchange. We use that in the persistent aggregator. So we should use that logic here as well.

        Long term the trick would be to survice server crashes, and support recovery when Camel comes back online, as the persistent store needs to keep state if the exchange has been redelivered or not. And how many redelivery attempts has already been made.
        The aggregator supports this.

        This logic should IMHO not be implemented in a patch release, and thus only applicable for 2.9.0

        Show
        davsclaus Claus Ibsen added a comment - There is some logic in DefaultExchangeHolder to serialize/unserialize a Camel Exchange. We use that in the persistent aggregator. So we should use that logic here as well. Long term the trick would be to survice server crashes, and support recovery when Camel comes back online, as the persistent store needs to keep state if the exchange has been redelivered or not. And how many redelivery attempts has already been made. The aggregator supports this. This logic should IMHO not be implemented in a patch release, and thus only applicable for 2.9.0
        Hide
        jbonofre Jean-Baptiste Onofré added a comment -

        Thanks for the explanation Claus.

        I will submit a patch for trunk, agree to provide this only on 2.9.0.

        Show
        jbonofre Jean-Baptiste Onofré added a comment - Thanks for the explanation Claus. I will submit a patch for trunk, agree to provide this only on 2.9.0.

          People

          • Assignee:
            Unassigned
            Reporter:
            jbonofre Jean-Baptiste Onofré
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development