Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1122

Kafka can log giant log lines

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 0.8.0
    • None
    • logging

    Description

      There are a number of log lines that the kafka server, and high-level consumer, can log, that can end up becoming a giant log line. This can be cumbersome to deal with in a log file.

      This happens in my case as I have have a large number of topics (on the order of 500-700 topics). Typically, these giant log lines will say something separately about every topic on the broker. An example:

      2013-11-04 23:28:11,148 INFO [kafka-request-handler-0] server.ReplicaManager - [Replica Manager on Broker 10]: Handling LeaderAndIsr request Name:LeaderAndIsrRequest;Version:0;Controller:11;ControllerEpoch:220;CorrelationId:5;ClientId:id_11-host_null-port_27330;PartitionState:(mytopic,0) -> (LeaderAndIsrInfo:(Leader:11,ISR:11,LeaderEpoch:43,ControllerEpoch:219),ReplicationFactor:2),.....

      Imagine that line going on with a separate entry for 700 topics. There are many other examples of this phenomenon in the server, and high-level consumer.

      I'd think these log lines could be separated into a single line per topic.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jbrosenberg@gmail.com Jason Rosenberg
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: