1. Kafka
  2. KAFKA-646

Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level


    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:


      WIth KAFKA-622, we measure ProducerRequestStats and FetchRequestAndResponseStats at the SyncProducer and SimpleConsumer level respectively. We could also aggregate them in the high level Producer and ZookeeperConsumerConnector level to provide an overall sense of request/response rate/size at the client level. Currently, I am not completely clear about the math that might be necessary for such aggregation or if metrics already provides an API for aggregating stats of the same type.

      We should also address the comments by Jun at KAFKA-622, I am copy pasting them here:

      60. What happens if have 2 instances of Consumers with the same clientid in the same jvm? Does one of them fail because it fails to register metrics? Ditto for Producers.
      61. ConsumerTopicStats: What if a topic is named AllTopics? We use to handle this by adding a - in topic specific stats.
      62. ZookeeperConsumerConnector: Do we need to validate groupid?
      63. ClientId: Does the clientid length need to be different from topic length?
      64. AbstractFetcherThread: When building a fetch request, do we need to pass in brokerInfo as part of the client id? BrokerInfo contains the source broker info and the fetch requests are always made to the source broker.

      1. kafka-646-patch-num1-v1.patch
        65 kB
        Swapnil Ghike
      2. kafka-646-patch-num1-v2.patch
        64 kB
        Swapnil Ghike
      3. kafka-646-patch-num1-v3.patch
        64 kB
        Swapnil Ghike
      4. kafka-646-patch-num1-v4.patch
        64 kB
        Swapnil Ghike
      5. kafka-646-patch-num1-v5.patch
        71 kB
        Swapnil Ghike
      6. kafka-646-patch-num1-v6.patch
        71 kB
        Swapnil Ghike
      7. kafka-646-patch-num1-v7.patch
        70 kB
        Swapnil Ghike



          • Assignee:
            Swapnil Ghike
            Swapnil Ghike
          • Votes:
            0 Vote for this issue
            3 Start watching this issue


            • Created: