Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3816

Provide more context in Kafka Connect log messages using MDC

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.9.0.1
    • Fix Version/s: 2.3.0
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      Currently it is relatively difficult to correlate individual log messages with the various threads and activities that are going on within a Kafka Connect worker, let along a cluster of workers. Log messages should provide more context to make it easier and to allow log scraping tools to coalesce related log messages.

      One simple way to do this is by using mapped diagnostic contexts, or MDC. This is supported by the SLF4J API, and by the Logback and Log4J logging frameworks.

      Basically, the framework would be changed so that each thread is configured with one or more MDC parameters using the org.slf4j.MDC.put(String,String) method in SLF4J. Once that thread is configured, all log messages made using that thread have that context. The logs can then be configured to use those parameters.

      It would be ideal to define a convention for connectors and the Kafka Connect framework. A single set of MDC parameters means that the logging framework can use the specific parameters on its message formats.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rhauch Randall Hauch
                Reporter:
                rhauch Randall Hauch
              • Votes:
                3 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: