[KAFKA-14742] Flaky ExactlyOnceSourceIntegrationTest.testConnectorBoundary OOMs

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Labels: flaky-test
• Assignee: Greg Harris
• Reporter: Greg Harris
• Reviewer: Edoardo Comar
• Created: 22/Feb/23 20:25
• Updated: 28/Feb/23 10:25
• Resolved: 28/Feb/23 10:25

Description

The ExactlyOnceSourceIntegrationTest appears to occasionally throw the following exception in my local test runs:

java.lang.OutOfMemoryError: GC overhead limit exceeded
	at java.base/java.util.HashMap.newNode(HashMap.java:1901)
	at java.base/java.util.HashMap.putVal(HashMap.java:629)
	at java.base/java.util.HashMap.put(HashMap.java:610)
	at java.base/java.util.HashSet.add(HashSet.java:221)
	at java.base/java.util.stream.Collectors$$Lambda$6/0x800000011.accept(Unknown Source)
	at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
	at java.base/java.util.stream.LongPipeline$1$1.accept(LongPipeline.java:177)
	at java.base/java.util.stream.Streams$RangeLongSpliterator.forEachRemaining(Streams.java:228)
	at java.base/java.util.Spliterator$OfLong.forEachRemaining(Spliterator.java:775)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest.lambda$assertSeqnos$9(ExactlyOnceSourceIntegrationTest.java:964)
	at org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest$$Lambda$2500/0x00000008015a1908.accept(Unknown Source)
	at java.base/java.util.HashMap.forEach(HashMap.java:1421)
	at org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest.assertSeqnos(ExactlyOnceSourceIntegrationTest.java:961)
	at org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest.assertExactlyOnceSeqnos(ExactlyOnceSourceIntegrationTest.java:939)
	at org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest.testIntervalBoundary(ExactlyOnceSourceIntegrationTest.java:358)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)

It appears that the data produced by the connectors under test is too large to be asserted on with the current assertions' memory overhead. We should try to optimize the assertions' overhead and or reduce the number of records being asserted on.

Issue Links

• links to: GitHub Pull Request #13291