Kafka
  1. Kafka
  2. KAFKA-440

Create a regression test framework for distributed environment testing

    Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None

      Description

      Initial requirements:

      1. The whole test framework is preferably coded in Python (a common scripting language which has well supported features)

      2. Test framework driver should be generic (distributed environment can be local host)

      3. Test framework related configurations are defined in JSON format

      4. Test environment, suite, case definitions may be defined in the following levels:
      4-a entity_id is used as a key for looking up related config from different levels

      4-b Cluster level defines: entity_id, hostname, kafka_home, java_home, ...

      4-c Test suite / case level defines:
      4-c-1 zookeeper: entity_id, clientPort, dataDir, log_filename, config_filename
      4-c-2 broker: entity_id, port, log.file.size, log.dir, log_filename, config_filename
      4-c-3 producer: entity_id, topic, threads, compression-codec, message-size, log_filename, config_filename

      1. kafka-440-v1.patch
        64 kB
        John Fung
      2. kafka-440-v2.patch
        72 kB
        John Fung
      3. kafka-440-v3.patch
        84 kB
        John Fung
      4. kafka-440-v5.patch
        89 kB
        John Fung
      5. kafka-440-v6.patch
        93 kB
        John Fung
      6. kafka-440-v7.patch
        96 kB
        John Fung

        Issue Links

        1.
        Simple message replication test Sub-task Closed John Fung

        0%

        Original Estimate - 48h
        Remaining Estimate - 48h
         
        2.
        Simple message replication test with varied producer acks Sub-task Closed John Fung

        0%

        Original Estimate - 24h
        Remaining Estimate - 24h
         
        3.
        Simple message replication test with multiple partitions Sub-task Closed John Fung

        0%

        Original Estimate - 24h
        Remaining Estimate - 24h
         
        4.
        Simple message replication test with multiple partitions and log segments Sub-task Closed John Fung

        0%

        Original Estimate - 24h
        Remaining Estimate - 24h
         
        5.
        Message replication with failures Sub-task Closed John Fung

        0%

        Original Estimate - 48h
        Remaining Estimate - 48h
         
        6.
        Message replication with failures and varied producer acks Sub-task Closed John Fung

        0%

        Original Estimate - 24h
        Remaining Estimate - 24h
         
        7.
        Message replication with failures and multiple partitions and log segments Sub-task Closed John Fung

        0%

        Original Estimate - 24h
        Remaining Estimate - 24h
         
        8.
        Message replication with failures and varied replication factor Sub-task Closed John Fung

        0%

        Original Estimate - 48h
        Remaining Estimate - 48h
         
        9.
        Leader election test Sub-task Closed John Fung

        0%

        Original Estimate - 24h
        Remaining Estimate - 24h
         
        10.
        Test in sync replica changes due to slow or stuck followers Sub-task Closed John Fung

        0%

        Original Estimate - 72h
        Remaining Estimate - 72h
         
        11. Refactor and optimize system tests Sub-task Open Unassigned  
         
        12.
        broker-list in testcase_1_properties.json should be picked up by the framework automatically Sub-task Resolved John Fung  
         
        13.
        Move start_consumer & start_producer inside "start_entity_in_background" Sub-task Resolved John Fung  
         
        14.
        Make producer to run for the entire duration of the System Test Sub-task Resolved John Fung  
         
        15.
        The system test is preferred to run out of the box without the need to update cluster_config.json Sub-task Resolved John Fung  
         
        16.
        Move log settings in the beginning of system_test_runner.py to an external log properties file Sub-task Resolved John Fung  
         
        17. Support MacOS for this test framework Sub-task Open John Fung  
         
        18.
        Rethrow exceptions to top level as much as possible Sub-task Resolved John Fung  
         
        19.
        Make sure all background running processes are terminated properly Sub-task Resolved John Fung  
         
        20.
        Port Mirroring System Test to this python system test framework Sub-task Resolved John Fung  
         
        21.
        Sometimes the python system test framework doesn't terminate all running processes Sub-task Resolved John Fung  
         
        22.
        Relative paths should be used for svg URLs in dashboards html Sub-task Resolved Unassigned  
         
        23.
        Simplify setup / initialization in replication_basic_test.py Sub-task Resolved John Fung  
         
        24.
        Support "testcase_to_run" or "testcase_to_skip" Sub-task Resolved John Fung  
         
        25.
        Shut down ZK last to avoid hanging brokers running processes Sub-task Resolved John Fung  
         
        26.
        Add more test cases to System Test Sub-task Closed John Fung  
         
        27. Add documentation for system tests Sub-task Open Unassigned  
         

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              John Fung
              Reporter:
              John Fung
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 360h
                360h
                Remaining:
                Remaining Estimate - 360h
                360h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development