Kafka
  1. Kafka
  2. KAFKA-263

Enhance single host broker failure test to have 2 topics with uneven distribution on the source brokers

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: clients
    • Labels:
      None

      Description

      The change is going to have 1 broker in the source cluster to consume messages of only 1 topic as shown below:

      source clusters
      ============
      broker1 - topic1, topic2
      broker2 - topic1, topic2
      broker3 - topic1, topic2
      broker4 - topic1

      1. kafka-263-v2.patch
        25 kB
        John Fung
      2. kafka-263-v1.patch
        25 kB
        John Fung

        Activity

        Hide
        Neha Narkhede added a comment -

        Committed this, after making the following minor changes -

        Changed ProducerPerformance to output the checksums at the DEBUG level
        Changed ProducerPerformance to use the new Logging trait.

        Show
        Neha Narkhede added a comment - Committed this, after making the following minor changes - Changed ProducerPerformance to output the checksums at the DEBUG level Changed ProducerPerformance to use the new Logging trait.
        Hide
        Neha Narkhede added a comment -

        +1 on v2. Thanks for working on this patch !

        Show
        Neha Narkhede added a comment - +1 on v2. Thanks for working on this patch !
        Hide
        John Fung added a comment -

        Hi Neha,

        Thanks for your review. I have made the changes in v2 patch:

        • init is reverted to its original signature
        • topic is passed into ChecksumMessageFormatter from props

        John

        Show
        John Fung added a comment - Hi Neha, Thanks for your review. I have made the changes in v2 patch: init is reverted to its original signature topic is passed into ChecksumMessageFormatter from props John
        Hide
        Neha Narkhede added a comment -

        I took a quick look at the patch and have one suggestion -

        • def init(props: Properties) {}
          + def init(props: Properties, tp: String) {}

        The above change seems unnecessary. Since the MessageFormatter takes in a Properties object, you can pass in the topic as part of the --property topic=foo argument for the ConsoleConsumer.

        Apart of the above, the patch looks good.

        Show
        Neha Narkhede added a comment - I took a quick look at the patch and have one suggestion - def init(props: Properties) {} + def init(props: Properties, tp: String) {} The above change seems unnecessary. Since the MessageFormatter takes in a Properties object, you can pass in the topic as part of the --property topic=foo argument for the ConsoleConsumer. Apart of the above, the patch looks good.
        Hide
        John Fung added a comment -

        Submitting patch. Please note that the following 2 files are modified to print topic info together with checksum such that the no. of checksum for each topic could be validated.

        core/src/main/scala/kafka/consumer/ConsoleConsumer.scala
        perf/src/main/scala/kafka/perf/ProducerPerformance.scala

        Show
        John Fung added a comment - Submitting patch. Please note that the following 2 files are modified to print topic info together with checksum such that the no. of checksum for each topic could be validated. core/src/main/scala/kafka/consumer/ConsoleConsumer.scala perf/src/main/scala/kafka/perf/ProducerPerformance.scala

          People

          • Assignee:
            John Fung
            Reporter:
            John Fung
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development