Kafka
  1. Kafka
  2. KAFKA-227

Add Broker Failure Test for a Single Host

    Details

    • Type: Test Test
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.1
    • Fix Version/s: 0.7.1
    • Component/s: clients
    • Labels:
      None

      Description

      Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.

      The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.

      Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.

      1. KAFKA-227-v3.patch
        68 kB
        John Fung
      2. KAFKA-227.patch
        67 kB
        John Fung

        Activity

        Hide
        John Fung added a comment -

        KAFKA-227-v3.patch has the following changes:

        1. Terminating processes cleanly
        2. Restarted Kafka broker log messages are appended to the log file of a previously terminated broker (instead of overwritten to)
        3. Report producer and source / mirror consumer message count for duplicate and missing messages
        4. Producer is running in the background
        5. Calling ConsumerOffsetChecker to wait for zero consumer offset lagging.

        Show
        John Fung added a comment - KAFKA-227 -v3.patch has the following changes: 1. Terminating processes cleanly 2. Restarted Kafka broker log messages are appended to the log file of a previously terminated broker (instead of overwritten to) 3. Report producer and source / mirror consumer message count for duplicate and missing messages 4. Producer is running in the background 5. Calling ConsumerOffsetChecker to wait for zero consumer offset lagging.
        Hide
        Neha Narkhede added a comment -

        This is a great patch ! Just one comment. The *sh scripts in the system_test/broker_failure need to be given execute permissions. Apart from that, it works well.

        Show
        Neha Narkhede added a comment - This is a great patch ! Just one comment. The *sh scripts in the system_test/broker_failure need to be given execute permissions. Apart from that, it works well.
        Hide
        Neha Narkhede added a comment -

        I made the svn propset changes and committed this patch. This system test has been instrumental in testing KAFKA-228 and I see how this can be useful in testing more Kafka bug fixes and features

        Show
        Neha Narkhede added a comment - I made the svn propset changes and committed this patch. This system test has been instrumental in testing KAFKA-228 and I see how this can be useful in testing more Kafka bug fixes and features

          People

          • Assignee:
            John Fung
            Reporter:
            John Fung
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development