[CAMEL-10586] make the kafka endpoint a little easier to use

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.19.0
• Component/s: camel-kafka
• Estimated Complexity: Unknown
• Assignee: Claus Ibsen
• Reporter: james strachan
• Created: 09/Dec/16 17:35
• Updated: 04/Mar/17 12:30
• Resolved: 04/Mar/17 12:22

Description

when using, say, JMS or activemq endpoints you can send text to a destination and it all kinda works OOTB. However it seems pretty hard to actually send anyhting to the kafka endpoint.

I wonder if we could do a bit more to make it easier to use as a camel endpoint?

e.g. if I invoke a HTTP endpoint then pipe the result to Kafka I get:

org.apache.kafka.common.errors.SerializationException: Can't convert value of class org.apache.camel.converter.stream.CachedOutputStream$WrappedInputStream to class org.apache.kafka.common.serialization.StringSerializer specified in value.serializer

seems like it'd be nice if there's no configured serializer to assume its bytes / string?

If I send to the dataformat endpoint first to force conversion to a String:

to("dataformat:string:unmarshal").
to("kafka://kafka:9092?topic=cheese&groupId=group1")

then I get:

17:34:10.212 [Camel (Funktion) thread #0 - timer://foo] WARN  o.a.c.component.kafka.KafkaProducer - No message key or partition key set

and there's no easy way to pass in a message key or partition key via the URI parameters from what I can see.

I wonder if we should support a URI parameter for setting the partition key + message key?