Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-10065

Support Iterator<...> for kafka component

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.16.4, 2.17.2
    • Component/s: camel-kafka
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      The kafka component currently takes the body of the message and sends that directly into the kafka producer then sets the callback for when that particular message is delivered to the kafka server. This works, but is slow, particularly for thousands of little messages. (cannot really take advantage of the batching that is done in the kafka producer)

      It would be good to allow the camel-kafka producer to accept various collections (Iterable and Iterator) and have it iterator through those creating kafka messages that can be uploaded and processed in a batch. dataformats (and aggregator) that can produce those could benefit tremendously.

        Attachments

          Activity

            People

            • Assignee:
              dkulp Daniel Kulp
              Reporter:
              dkulp Daniel Kulp
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: