Camel
  1. Camel
  2. CAMEL-4575

Persistant Dead Letter Channel to avoid memory consumption between retries

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.0.0, Future
    • Component/s: camel-core
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      When using a DeadLetterChannel, the messages are stored in memory between the retries (redeliveries). They are flushed from the memory only when we get the maximumRedeliveries value.

      It means that we can have an important memory consumption, because the messages are memory resident for a long time when:

      • if we have an important maximumRedeliveries, especially if we have -1
      • if we have an important redeliveryDelay

      I propose to create a PersistentDeadLetterChannel, working like the DeadLetterChannel, but, between redeliveries, the messages are flushed to a persistent store (filesystem, JMQ queue, JDBC, ...).

        Activity

        Hide
        Claus Ibsen added a comment -

        There is some logic in DefaultExchangeHolder to serialize/unserialize a Camel Exchange. We use that in the persistent aggregator. So we should use that logic here as well.

        Long term the trick would be to survice server crashes, and support recovery when Camel comes back online, as the persistent store needs to keep state if the exchange has been redelivered or not. And how many redelivery attempts has already been made.
        The aggregator supports this.

        This logic should IMHO not be implemented in a patch release, and thus only applicable for 2.9.0

        Show
        Claus Ibsen added a comment - There is some logic in DefaultExchangeHolder to serialize/unserialize a Camel Exchange. We use that in the persistent aggregator. So we should use that logic here as well. Long term the trick would be to survice server crashes, and support recovery when Camel comes back online, as the persistent store needs to keep state if the exchange has been redelivered or not. And how many redelivery attempts has already been made. The aggregator supports this. This logic should IMHO not be implemented in a patch release, and thus only applicable for 2.9.0
        Hide
        Jean-Baptiste Onofré added a comment -

        Thanks for the explanation Claus.

        I will submit a patch for trunk, agree to provide this only on 2.9.0.

        Show
        Jean-Baptiste Onofré added a comment - Thanks for the explanation Claus. I will submit a patch for trunk, agree to provide this only on 2.9.0.

          People

          • Assignee:
            Unassigned
            Reporter:
            Jean-Baptiste Onofré
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development