Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-25456

FlinkKafkaProducerITCase.testScaleDownBeforeFirstCheckpoint

    XMLWordPrintableJSON

Details

    Description

      The test FlinkKafkaProducerITCase.testScaleDownBeforeFirstCheckpoint fails with

      2021-12-27T02:54:54.8464375Z Dec 27 02:54:54 [ERROR] Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 285.279 s <<< FAILURE! - in org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase
2021-12-27T02:54:54.8465354Z Dec 27 02:54:54 [ERROR] testScaleDownBeforeFirstCheckpoint  Time elapsed: 85.514 s  <<< FAILURE!
2021-12-27T02:54:54.8468827Z Dec 27 02:54:54 java.lang.AssertionError: Detected producer leak. Thread name: kafka-producer-network-thread | producer-MockTask-000000000000002a000000000000002c-18
2021-12-27T02:54:54.8469779Z Dec 27 02:54:54 	at org.junit.Assert.fail(Assert.java:89)
2021-12-27T02:54:54.8470485Z Dec 27 02:54:54 	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase.checkProducerLeak(FlinkKafkaProducerITCase.java:847)
2021-12-27T02:54:54.8471842Z Dec 27 02:54:54 	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase.testScaleDownBeforeFirstCheckpoint(FlinkKafkaProducerITCase.java:381)
2021-12-27T02:54:54.8472724Z Dec 27 02:54:54 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2021-12-27T02:54:54.8473509Z Dec 27 02:54:54 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2021-12-27T02:54:54.8474704Z Dec 27 02:54:54 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2021-12-27T02:54:54.8475523Z Dec 27 02:54:54 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2021-12-27T02:54:54.8476258Z Dec 27 02:54:54 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
2021-12-27T02:54:54.8476949Z Dec 27 02:54:54 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
2021-12-27T02:54:54.8477632Z Dec 27 02:54:54 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
2021-12-27T02:54:54.8478451Z Dec 27 02:54:54 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
2021-12-27T02:54:54.8479282Z Dec 27 02:54:54 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
2021-12-27T02:54:54.8479976Z Dec 27 02:54:54 	at org.apache.flink.testutils.junit.RetryRule$RetryOnFailureStatement.evaluate(RetryRule.java:135)
2021-12-27T02:54:54.8480696Z Dec 27 02:54:54 	at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
2021-12-27T02:54:54.8481410Z Dec 27 02:54:54 	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
2021-12-27T02:54:54.8482009Z Dec 27 02:54:54 	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
2021-12-27T02:54:54.8482636Z Dec 27 02:54:54 	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
2021-12-27T02:54:54.8483267Z Dec 27 02:54:54 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
2021-12-27T02:54:54.8483900Z Dec 27 02:54:54 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
2021-12-27T02:54:54.8484574Z Dec 27 02:54:54 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
2021-12-27T02:54:54.8485214Z Dec 27 02:54:54 	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
2021-12-27T02:54:54.8485838Z Dec 27 02:54:54 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
2021-12-27T02:54:54.8486441Z Dec 27 02:54:54 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
2021-12-27T02:54:54.8487037Z Dec 27 02:54:54 	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
2021-12-27T02:54:54.8487620Z Dec 27 02:54:54 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
2021-12-27T02:54:54.8488391Z Dec 27 02:54:54 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
2021-12-27T02:54:54.8489050Z Dec 27 02:54:54 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
2021-12-27T02:54:54.8489685Z Dec 27 02:54:54 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
2021-12-27T02:54:54.8490269Z Dec 27 02:54:54 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
2021-12-27T02:54:54.8490844Z Dec 27 02:54:54 	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
2021-12-27T02:54:54.8491498Z Dec 27 02:54:54 	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
2021-12-27T02:54:54.8492038Z Dec 27 02:54:54 	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
2021-12-27T02:54:54.8492578Z Dec 27 02:54:54 	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
2021-12-27T02:54:54.8493179Z Dec 27 02:54:54 	at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43)
2021-12-27T02:54:54.8493836Z Dec 27 02:54:54 	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
2021-12-27T02:54:54.8494493Z Dec 27 02:54:54 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
2021-12-27T02:54:54.8495111Z Dec 27 02:54:54 	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
2021-12-27T02:54:54.8495760Z Dec 27 02:54:54 	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
2021-12-27T02:54:54.8496418Z Dec 27 02:54:54 	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
2021-12-27T02:54:54.8497202Z Dec 27 02:54:54 	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
2021-12-27T02:54:54.8497877Z Dec 27 02:54:54 	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
2021-12-27T02:54:54.8498721Z Dec 27 02:54:54 	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
2021-12-27T02:54:54.8499399Z Dec 27 02:54:54 	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2021-12-27T02:54:54.8500050Z Dec 27 02:54:54 	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
2021-12-27T02:54:54.8500751Z Dec 27 02:54:54 	at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:82)
2021-12-27T02:54:54.8501613Z Dec 27 02:54:54 	at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:73)
2021-12-27T02:54:54.8502275Z Dec 27 02:54:54 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:220)
2021-12-27T02:54:54.8502962Z Dec 27 02:54:54 	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:188)
2021-12-27T02:54:54.8503692Z Dec 27 02:54:54 	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202)
2021-12-27T02:54:54.8504405Z Dec 27 02:54:54 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181)
2021-12-27T02:54:54.8505078Z Dec 27 02:54:54 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
2021-12-27T02:54:54.8505813Z Dec 27 02:54:54 	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:150)
2021-12-27T02:54:54.8506580Z Dec 27 02:54:54 	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:120)
2021-12-27T02:54:54.8507331Z Dec 27 02:54:54 	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
2021-12-27T02:54:54.8508149Z Dec 27 02:54:54 	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
2021-12-27T02:54:54.8508880Z Dec 27 02:54:54 	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
2021-12-27T02:54:54.8509523Z Dec 27 02:54:54 	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
2021-12-27T02:54:54.8509996Z Dec 27 02:54:54 
2021-12-27T02:54:55.3284748Z Dec 27 02:54:55 [INFO] 
2021-12-27T02:54:55.3285368Z Dec 27 02:54:55 [INFO] Results:
2021-12-27T02:54:55.3285735Z Dec 27 02:54:55 [INFO] 
2021-12-27T02:54:55.3286103Z Dec 27 02:54:55 [ERROR] Failures: 
2021-12-27T02:54:55.3287741Z Dec 27 02:54:55 [ERROR]   FlinkKafkaProducerITCase.testScaleDownBeforeFirstCheckpoint:381->checkProducerLeak:847 Detected producer leak. Thread name: kafka-producer-network-thread | producer-MockTask-000000000000002a000000000000002c-18

      The problem seems to be that a producer leak has been detected (cc fpaul).

      https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28602&view=logs&j=ce8f3cc3-c1ea-5281-f5eb-df9ebd24947f&t=918e890f-5ed9-5212-a25e-962628fb4bc5&l=7186

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. FLINK-22342 FlinkKafkaProducerITCase fails with producer leak

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. FLINK-22056 FlinkKafkaProducerITCase failed because of "producer leak".

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              Unassigned Unassigned
              trohrmann Till Rohrmann
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: