Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-15140

Improve TopicCommandIntegrationTest to be less flaky



    • Test
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 3.8.0
    • unit tests


      This is a good Jira for folks who are new to contributing to Kafka.

      Tests in TopicCommandIntegrationTest get flaky from time to time. The objective of the task is to make them more robust by doing the following:

      1. Replace the usage createAndWaitTopic() adminClient.createTopics() method and other places where were are creating a topic (without waiting) with 
      TestUtils.createTopicWithAdmin(). The latter method already contains the functionality to create a topic and wait for metadata to sync up.

      2. Replace the number 6 at places such as 
      Collections.singletonList(new NewTopic("foo_bar", 1, 6.toShort)))" with a meaningful constant.

      3. Add logs if an assertion fails, for example, lines such as "
      assertTrue(rows(0).startsWith(s"\tTopic: $testTopicName"), output)" should have a third argument which prints the actual output printed so that we can observe in the test logs on what was the output when assertion failed.

      4. Replace occurrences of "\n" with System.lineSeparator() which is platform independent

      5. We should wait for reassignment to complete whenever we are re-assigning partitions using alterconfig before we call describe to validate it. We could use 

      Motivation of this task
      Try to fix the flaky test behaviour such as observed in https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13924/5/testReport/junit/kafka.admin/TopicCommandIntegrationTest/Build___JDK_11_and_Scala_2_13___testDescribeUnderMinIsrPartitionsMixed_String__quorum_zk/ 

      org.opentest4j.AssertionFailedError: expected: <true> but was: <false>
      	at app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
      	at app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
      	at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
      	at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
      	at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:31)
      	at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:180)
      	at app//kafka.admin.TopicCommandIntegrationTest.testDescribeUnderMinIsrPartitionsMixed(TopicCommandIntegrationTest.scala:794)




            isding_l Lan Ding
            divijvaidya Divij Vaidya
            Divij Vaidya Divij Vaidya
            0 Vote for this issue
            7 Start watching this issue