[FLINK-18444] KafkaITCase failing with "Failed to send data to Kafka: This server does not host this topic-partition"

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.11.3, 1.12.0
• Component/s: Connectors / Kafka, Tests
• Labels: auto-deprioritized-critical auto-deprioritized-major stale-assigned test-stability
• Assignee: Fabian Paul
• Reporter: Dian Fu
• Created: 29/Jun/20 02:52
• Updated: 01/Jan/22 22:38

Description

Instance on master: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=4092&view=logs&j=ce8f3cc3-c1ea-5281-f5eb-df9ebd24947f

2020-06-28T21:37:54.8113215Z [ERROR] testMultipleSourcesOnePartition(org.apache.flink.streaming.connectors.kafka.KafkaITCase)  Time elapsed: 5.079 s  <<< ERROR!
2020-06-28T21:37:54.8113885Z org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
2020-06-28T21:37:54.8114418Z 	at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:147)
2020-06-28T21:37:54.8114905Z 	at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:677)
2020-06-28T21:37:54.8115397Z 	at org.apache.flink.streaming.util.TestStreamEnvironment.execute(TestStreamEnvironment.java:81)
2020-06-28T21:37:54.8116254Z 	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1699)
2020-06-28T21:37:54.8116857Z 	at org.apache.flink.streaming.connectors.kafka.testutils.DataGenerators.generateRandomizedIntegerSequence(DataGenerators.java:120)
2020-06-28T21:37:54.8117715Z 	at org.apache.flink.streaming.connectors.kafka.KafkaConsumerTestBase.runMultipleSourcesOnePartitionExactlyOnceTest(KafkaConsumerTestBase.java:933)
2020-06-28T21:37:54.8118327Z 	at org.apache.flink.streaming.connectors.kafka.KafkaITCase.testMultipleSourcesOnePartition(KafkaITCase.java:107)
2020-06-28T21:37:54.8118805Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2020-06-28T21:37:54.8119859Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2020-06-28T21:37:54.8120861Z 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2020-06-28T21:37:54.8121436Z 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2020-06-28T21:37:54.8121899Z 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
2020-06-28T21:37:54.8122424Z 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
2020-06-28T21:37:54.8122942Z 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
2020-06-28T21:37:54.8123406Z 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
2020-06-28T21:37:54.8123899Z 	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
2020-06-28T21:37:54.8124507Z 	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
2020-06-28T21:37:54.8124978Z 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
2020-06-28T21:37:54.8125332Z 	at java.base/java.lang.Thread.run(Thread.java:834)
2020-06-28T21:37:54.8125743Z Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
2020-06-28T21:37:54.8126305Z 	at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:116)
2020-06-28T21:37:54.8126961Z 	at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:78)
2020-06-28T21:37:54.8127766Z 	at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:192)
2020-06-28T21:37:54.8128570Z 	at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:185)
2020-06-28T21:37:54.8129140Z 	at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:179)
2020-06-28T21:37:54.8129686Z 	at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:506)
2020-06-28T21:37:54.8130174Z 	at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:386)
2020-06-28T21:37:54.8130717Z 	at jdk.internal.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
2020-06-28T21:37:54.8131254Z 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2020-06-28T21:37:54.8131954Z 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2020-06-28T21:37:54.8132626Z 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:284)
2020-06-28T21:37:54.8133434Z 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:199)
2020-06-28T21:37:54.8134163Z 	at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
2020-06-28T21:37:54.8134728Z 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152)
2020-06-28T21:37:54.8135163Z 	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
2020-06-28T21:37:54.8135555Z 	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
2020-06-28T21:37:54.8145600Z 	at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
2020-06-28T21:37:54.8146039Z 	at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
2020-06-28T21:37:54.8146569Z 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
2020-06-28T21:37:54.8146974Z 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
2020-06-28T21:37:54.8147737Z 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
2020-06-28T21:37:54.8148145Z 	at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
2020-06-28T21:37:54.8148529Z 	at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
2020-06-28T21:37:54.8148899Z 	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
2020-06-28T21:37:54.8149248Z 	at akka.actor.ActorCell.invoke(ActorCell.scala:561)
2020-06-28T21:37:54.8149580Z 	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
2020-06-28T21:37:54.8149919Z 	at akka.dispatch.Mailbox.run(Mailbox.scala:225)
2020-06-28T21:37:54.8150236Z 	at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
2020-06-28T21:37:54.8150593Z 	at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
2020-06-28T21:37:54.8151016Z 	at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
2020-06-28T21:37:54.8151432Z 	at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
2020-06-28T21:37:54.8151868Z 	at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2020-06-28T21:37:54.8153483Z Caused by: java.util.concurrent.ExecutionException: org.apache.flink.streaming.connectors.kafka.FlinkKafkaException: Failed to send data to Kafka: Failed to send data to Kafka: This server does not host this topic-partition.
2020-06-28T21:37:54.8154221Z 	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
2020-06-28T21:37:54.8154690Z 	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2022)
2020-06-28T21:37:54.8155228Z 	at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.quiesceTimeServiceAndCloseOperator(StreamOperatorWrapper.java:161)
2020-06-28T21:37:54.8155815Z 	at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.close(StreamOperatorWrapper.java:130)
2020-06-28T21:37:54.8156615Z 	at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.close(StreamOperatorWrapper.java:80)
2020-06-28T21:37:54.8157189Z 	at org.apache.flink.streaming.runtime.tasks.OperatorChain.closeOperators(OperatorChain.java:302)
2020-06-28T21:37:54.8157863Z 	at org.apache.flink.streaming.runtime.tasks.StreamTask.afterInvoke(StreamTask.java:569)
2020-06-28T21:37:54.8158317Z 	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:538)
2020-06-28T21:37:54.8158747Z 	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
2020-06-28T21:37:54.8159133Z 	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
2020-06-28T21:37:54.8159473Z 	at java.base/java.lang.Thread.run(Thread.java:834)
2020-06-28T21:37:54.8160355Z Caused by: org.apache.flink.streaming.connectors.kafka.FlinkKafkaException: Failed to send data to Kafka: Failed to send data to Kafka: This server does not host this topic-partition.
2020-06-28T21:37:54.8161098Z 	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.checkErroneous(FlinkKafkaProducer.java:1284)
2020-06-28T21:37:54.8161892Z 	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.close(FlinkKafkaProducer.java:908)
2020-06-28T21:37:54.8162646Z 	at org.apache.flink.api.common.functions.util.FunctionUtils.closeFunction(FunctionUtils.java:43)
2020-06-28T21:37:54.8163511Z 	at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.close(AbstractUdfStreamOperator.java:109)
2020-06-28T21:37:54.8164383Z 	at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.lambda$closeOperator$5(StreamOperatorWrapper.java:205)
2020-06-28T21:37:54.8165456Z 	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
2020-06-28T21:37:54.8166757Z 	at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.closeOperator(StreamOperatorWrapper.java:203)
2020-06-28T21:37:54.8167837Z 	at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.lambda$deferCloseOperatorToMailbox$3(StreamOperatorWrapper.java:177)
2020-06-28T21:37:54.8168498Z 	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
2020-06-28T21:37:54.8169014Z 	at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:78)
2020-06-28T21:37:54.8169501Z 	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxExecutorImpl.tryYield(MailboxExecutorImpl.java:91)
2020-06-28T21:37:54.8170085Z 	at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.quiesceTimeServiceAndCloseOperator(StreamOperatorWrapper.java:155)
2020-06-28T21:37:54.8170496Z 	... 8 more
2020-06-28T21:37:54.8171421Z Caused by: org.apache.flink.streaming.connectors.kafka.FlinkKafkaException: Failed to send data to Kafka: This server does not host this topic-partition.
2020-06-28T21:37:54.8172062Z 	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.checkErroneous(FlinkKafkaProducer.java:1284)
2020-06-28T21:37:54.8172665Z 	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.flush(FlinkKafkaProducer.java:1030)
2020-06-28T21:37:54.8173251Z 	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.close(FlinkKafkaProducer.java:872)
2020-06-28T21:37:54.8173592Z 	... 18 more
2020-06-28T21:37:54.8174191Z Caused by: org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition.

Issue Links

• is related to: FLINK-12030 KafkaITCase.testMultipleSourcesOnePartition is unstable: This server does not host this topic-partition