Kafka
  1. Kafka
  2. KAFKA-227

Add Broker Failure Test for a Single Host

    Details

    • Type: Test Test
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.1
    • Fix Version/s: 0.7.1
    • Component/s: clients
    • Labels:
      None

      Description

      Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.

      The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.

      Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.

      1. KAFKA-227.patch
        67 kB
        John Fung
      2. KAFKA-227-v3.patch
        68 kB
        John Fung

        Activity

        John Fung created issue -
        John Fung made changes -
        Field Original Value New Value
        Attachment KAFKA-227.patch [ 12507725 ]
        John Fung made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        John Fung added a comment -

        KAFKA-227-v3.patch has the following changes:

        1. Terminating processes cleanly
        2. Restarted Kafka broker log messages are appended to the log file of a previously terminated broker (instead of overwritten to)
        3. Report producer and source / mirror consumer message count for duplicate and missing messages
        4. Producer is running in the background
        5. Calling ConsumerOffsetChecker to wait for zero consumer offset lagging.

        Show
        John Fung added a comment - KAFKA-227 -v3.patch has the following changes: 1. Terminating processes cleanly 2. Restarted Kafka broker log messages are appended to the log file of a previously terminated broker (instead of overwritten to) 3. Report producer and source / mirror consumer message count for duplicate and missing messages 4. Producer is running in the background 5. Calling ConsumerOffsetChecker to wait for zero consumer offset lagging.
        John Fung made changes -
        Attachment KAFKA-227-v3.patch [ 12509598 ]
        Hide
        Neha Narkhede added a comment -

        This is a great patch ! Just one comment. The *sh scripts in the system_test/broker_failure need to be given execute permissions. Apart from that, it works well.

        Show
        Neha Narkhede added a comment - This is a great patch ! Just one comment. The *sh scripts in the system_test/broker_failure need to be given execute permissions. Apart from that, it works well.
        Hide
        Neha Narkhede added a comment -

        I made the svn propset changes and committed this patch. This system test has been instrumental in testing KAFKA-228 and I see how this can be useful in testing more Kafka bug fixes and features

        Show
        Neha Narkhede added a comment - I made the svn propset changes and committed this patch. This system test has been instrumental in testing KAFKA-228 and I see how this can be useful in testing more Kafka bug fixes and features
        Neha Narkhede made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        2m 24s 1 John Fung 16/Dec/11 21:09
        Patch Available Patch Available Resolved Resolved
        20d 3h 36m 1 Neha Narkhede 06/Jan/12 00:45

          People

          • Assignee:
            John Fung
            Reporter:
            John Fung
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development