[KAFKA-12566] Flaky Test MirrorConnectorsIntegrationSSLTest#testReplication

• Type: Test
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.5.0, 3.4.1, 3.3.3
• Component/s: mirrormaker, unit tests
• Labels: flaky-test
• Assignee: Greg Harris
• Reporter: Matthias J. Sax
• Created: 27/Mar/21 18:10
• Updated: 07/Jun/23 20:25
• Resolved: 17/Feb/23 22:25

Description

 

org.opentest4j.AssertionFailedError: Condition not met within timeout 20000. Offsets not translated downstream to primary cluster. ==> expected: <true> but was: <false> at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55) at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:40) at org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:193) at org.apache.kafka.test.TestUtils.lambda$waitForCondition$3(TestUtils.java:303) at org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:351) at org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:319) at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:300) at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:290) at org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.testReplication(MirrorConnectorsIntegrationBaseTest.java:289)

LOGs

[2021-03-26 03:28:06,157] ERROR Could not check connector state info. (org.apache.kafka.connect.util.clusters.EmbeddedConnectClusterAssertions:420) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: Could not read connector state. Error response: {"error_code":404,"message":"No status found for connector MirrorSourceConnector"} at org.apache.kafka.connect.util.clusters.EmbeddedConnectCluster.connectorStatus(EmbeddedConnectCluster.java:479) at org.apache.kafka.connect.util.clusters.EmbeddedConnectClusterAssertions.checkConnectorState(EmbeddedConnectClusterAssertions.java:413) at org.apache.kafka.connect.util.clusters.EmbeddedConnectClusterAssertions.lambda$assertConnectorAndAtLeastNumTasksAreRunning$16(EmbeddedConnectClusterAssertions.java:286) at org.apache.kafka.test.TestUtils.lambda$waitForCondition$3(TestUtils.java:303) at org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:351) at org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:319) at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:300) at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:290) at org.apache.kafka.connect.util.clusters.EmbeddedConnectClusterAssertions.assertConnectorAndAtLeastNumTasksAreRunning(EmbeddedConnectClusterAssertions.java:285) at org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.waitUntilMirrorMakerIsRunning(MirrorConnectorsIntegrationBaseTest.java:470) at org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.testReplication(MirrorConnectorsIntegrationBaseTest.java:227)

and

[2021-03-26 03:30:41,524] ERROR [MirrorHeartbeatConnector|task-0] Graceful stop of task MirrorHeartbeatConnector-0 failed. (org.apache.kafka.connect.runtime.Worker:866) [2021-03-26 03:30:41,527] ERROR [MirrorHeartbeatConnector|task-0] WorkerSourceTask{id=MirrorHeartbeatConnector-0} failed to send record to heartbeats: (org.apache.kafka.connect.runtime.WorkerSourceTask:372) org.apache.kafka.common.KafkaException: Producer is closed forcefully. at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortBatches(RecordAccumulator.java:750) at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortIncompleteBatches(RecordAccumulator.java:737) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:282) at java.lang.Thread.run(Thread.java:748) [2021-03-26 03:30:42,248] ERROR [MirrorHeartbeatConnector|task-0] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Failed to flush, timed out while waiting for producer to flush outstanding 1 messages (org.apache.kafka.connect.runtime.WorkerSourceTask:512)

 

Issue Links

• is related to: KAFKA-15072 Flaky test MirrorConnectorsIntegrationExactlyOnceTest.testReplicationWithEmptyPartition
• relates to: KAFKA-9013 Flaky Test MirrorConnectorsIntegrationTest#testReplication
• links to: GitHub Pull Request #13178