Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-16064

camel-kafka - Add batching consumer

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.10.0
    • Component/s: camel-kafka
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      It would be good to make camel-kafka and camel-vertx-kafka consumer able to batch together N messages via aggregation strategy as a single outgoing exchange. The batch should support group size, and timeout.

      Something that was in the sjms-batch component.

      The batching should use the same KafkaConsumer instance and defer the commit till the batching has been done. This requires manual offset commit by the consumer.

      The use-case has been discussed on the chat at
      https://camel.zulipchat.com/#narrow/stream/257298-camel/topic/kafka.20and.20batching

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              davsclaus Claus Ibsen
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: