Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-16815

OpenTracing with Avro keys causes warning

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 3.11.1, 3.12.0
    • camel-opentracing
    • None
    • Unknown

    Description

      I am writing an event to Kafka. The key and value are both written in Avro format (same entity used). Code is like:

      // invokation: curl -H "Content-Type: application/json" -X POST -d '{"id": "1", "message": "some message text"}' localhost:8080/event
      from(platformHttp("/event").httpMethodRestrict(HttpMethod.POST))
                      .log("Start")
                      .unmarshal().json(JsonLibrary.Jackson, AvroEntry.class)
      
                      // set avro header
                      .process(exchange -> {
                          exchange.getIn().setHeader(KafkaConstants.KEY, new AvroEntry(1L, "header"));
                      })
      
                      .to(kafka("{{kafka.to.topic}}")
                          .brokers("{{kafka.bootstrap.servers}}")
                          .securityProtocol("{{kafka.security.protocol}}")
                          .sslContextParameters(getSslContext())
                          .sslKeystoreType("{{kafka.ssl.keystore.type}}")
                          .sslTruststoreType("{{kafka.ssl.truststore.type}}")
                          .schemaRegistryURL("{{schema.registry.url}}")
                          .valueSerializer("{{kafka.value.serializer}}")
                          .keySerializer("{{kafka.key.serializer}}"))
      
                      // Set response for invoker
                      .setBody(constant("{\"message\": \"done\"}"))
                      .log("End");
      

      When OpenTracing is included, the following warning is shown:

      2021-07-19 14:53:39,648 WARN  [org.apa.cam.tra.Tracer] (vert.x-worker-thread-1) Tracing: Failed to capture tracing data: java.lang.ClassCastException: class com.example.model.AvroEntry cannot be cast to class java.lang.String (com.example.model.AvroEntry is in unnamed module of loader io.quarkus.bootstrap.classloading.QuarkusClassLoader @46dd830d; java.lang.String is in module java.base of loader 'bootstrap')
              at org.apache.camel.tracing.decorators.KafkaSpanDecorator.pre(KafkaSpanDecorator.java:75)
              at org.apache.camel.tracing.Tracer$TracingEventNotifier.notify(Tracer.java:246)
              at org.apache.camel.support.EventHelper.doNotifyEvent(EventHelper.java:1240)
              at org.apache.camel.support.EventHelper.notifyExchangeSending(EventHelper.java:704)
              at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:140)
              at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:395)
              at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148)
              at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60)
              at org.apache.camel.processor.Pipeline.process(Pipeline.java:147)
              at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:312)
              at org.apache.camel.component.platform.http.vertx.VertxPlatformHttpConsumer.lambda$handleRequest$2(VertxPlatformHttpConsumer.java:184)
              at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(ContextImpl.java:313)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
              at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
              at java.base/java.lang.Thread.run(Thread.java:829)
      

      This comprehensive warning is written for every message.

       

      However, the message is still written to Kafka. Seems like the tracer can not handle a non-string Key (https://github.com/apache/camel/blob/80b92e3624ae5db59a1a24a441f1b10b39eaa1a5/components/camel-tracing/src/main/java/org/apache/camel/tracing/decorators/KafkaSpanDecorator.java#L75). 

      Furthermore, when using Avro format as message key, the uber-trace-id in the message headers is missing. See the comparision (kafkacat extract):

      // When using no key
      Topic: avro-topic
         Key (-1 bytes):
         Value (24 bytes): "some message text
         Headers: Accept=*/*,Content-Length=43,Content-Type=application/json,Host=localhost:8080,uber-trace-id=a5c968dea6f3d52c:bfdfb86ec15171db:a5c968dea6f3d52c:0,User-Agent=curl/7.71.1
         Timestamp: 1626696278639
         Partition: 0
         Offset: 0
      
      // When using Avro key (no trace info)
      Topic: avro-topic
         Key (13 bytes): header
         Value (24 bytes): "some message text
         Headers: Accept=*/*,Content-Length=43,Content-Type=application/json,Host=localhost:8080,User-Agent=curl/7.71.1
         Timestamp: 1626696849111
         Partition: 0
         Offset: 1
      

      See also this discussion on zulip. 

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            davsclaus Claus Ibsen
            Stuber Thomas Stuber
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment