Kafka
  1. Kafka
  2. KAFKA-674

Clean Shutdown Testing - Log segments checksums mismatch

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None

      Activity

      Hide
      John Fung added a comment -

      Uploaded kafka-674-reproduce-issue-v2.patch with the following changes:
      1. Change the topics no. from 30 to 50 in testcase_9062
      2. Rebased with the latest 0.8 branch

      Show
      John Fung added a comment - Uploaded kafka-674-reproduce-issue-v2.patch with the following changes: 1. Change the topics no. from 30 to 50 in testcase_9062 2. Rebased with the latest 0.8 branch
      Hide
      John Fung added a comment -

      Test description:
      ============
      1. Start 1 Zookeeper
      2. Start a 4-broker cluster (Replication factor = 4, 2 partitions)
      3. Keep sending messages to 30 topics by ProducerPerformance
      4. Perform rolling bounce on all brokers (Controller is the last broker to bounce)
      5. Stop ProducerPerformance
      6. Start ConsoleConsumer
      7. Validate MessageID between Producer side & Consumer side (no data loss in all topics)
      8. Validate log segment checksums across all replicas:

      [/tmp] l kafka_server_1_logs/topic_0001-0/
      rw-rr- 1 jfung eng 416 Dec 13 11:18 00000000000000000000.index
      rw-rr- 1 jfung eng 298768 Dec 13 11:17 00000000000000000000.log

      [/tmp] l kafka_server_2_logs/topic_0001-0/
      rw-rr- 1 jfung eng 632 Dec 13 11:19 00000000000000000000.index
      rw-rr- 1 jfung eng 561768 Dec 13 11:18 00000000000000000000.log

      9. If the no. of topics is changed to 20, there will be no issue.

      To run the test:
      ============
      1. Apply the attached patch (no need to rebuild Kafka)
      2. Under <Kafka_home>/system_test, run this command:
      $ python -u -B system_test_runner.py 2>&1 | tee system_test_output_`date +%s`.log
      3. It will execute testcase_9062
      4. Data log will be located in /tmp/kafka_server_*_logs
      5. Log4j messages logs will be under <kafka_home>/system_test/clean_shutdown_testsuite/testcase_9062/logs

      Show
      John Fung added a comment - Test description: ============ 1. Start 1 Zookeeper 2. Start a 4-broker cluster (Replication factor = 4, 2 partitions) 3. Keep sending messages to 30 topics by ProducerPerformance 4. Perform rolling bounce on all brokers (Controller is the last broker to bounce) 5. Stop ProducerPerformance 6. Start ConsoleConsumer 7. Validate MessageID between Producer side & Consumer side (no data loss in all topics) 8. Validate log segment checksums across all replicas: [/tmp] l kafka_server_1_logs/topic_0001-0/ rw-r r - 1 jfung eng 416 Dec 13 11:18 00000000000000000000.index rw-r r - 1 jfung eng 298768 Dec 13 11:17 00000000000000000000.log [/tmp] l kafka_server_2_logs/topic_0001-0/ rw-r r - 1 jfung eng 632 Dec 13 11:19 00000000000000000000.index rw-r r - 1 jfung eng 561768 Dec 13 11:18 00000000000000000000.log 9. If the no. of topics is changed to 20, there will be no issue. To run the test: ============ 1. Apply the attached patch (no need to rebuild Kafka) 2. Under <Kafka_home>/system_test, run this command: $ python -u -B system_test_runner.py 2>&1 | tee system_test_output_`date +%s`.log 3. It will execute testcase_9062 4. Data log will be located in /tmp/kafka_server_*_logs 5. Log4j messages logs will be under <kafka_home>/system_test/clean_shutdown_testsuite/testcase_9062/logs

        People

        • Assignee:
          Unassigned
          Reporter:
          John Fung
        • Votes:
          0 Vote for this issue
          Watchers:
          1 Start watching this issue

          Dates

          • Created:
            Updated:

            Development