[FLINK-25740] PulsarSourceOrderedE2ECase fails on azure

• Type: Sub-task
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.16.0, 1.15.3
• Component/s: Connectors / Pulsar
• Labels: stale-assigned test-stability
• Assignee: Yufei Zhang
• Reporter: Roman Khachatryan
• Created: 20/Jan/22 22:36
• Updated: 30/Aug/22 11:39
• Resolved: 30/Aug/22 11:39

Description

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29789&view=logs&j=af184cdd-c6d8-5084-0b69-7e9c67b35f7a&t=160c9ae5-96fd-516e-1c91-deb81f59292a&l=16385

[ERROR] Errors:
[ERROR]   PulsarSourceOrderedE2ECase>SourceTestSuiteBase.testIdleReader:187->SourceTestSuiteBase.gene rateAndWriteTestData:315 » BrokerPersistence
[ERROR]   PulsarSourceOrderedE2ECase>SourceTestSuiteBase.testIdleReader:187->SourceTestSuiteBase.gene rateAndWriteTestData:315 » BrokerPersistence
[ERROR]   PulsarSourceOrderedE2ECase>SourceTestSuiteBase.testMultipleSplits:145->SourceTestSuiteBase. generateAndWriteTestData:315 » BrokerPersistence
[ERROR]   PulsarSourceOrderedE2ECase>SourceTestSuiteBase.testMultipleSplits:145->SourceTestSuiteBase. generateAndWriteTestData:315 » BrokerPersistence
[ERROR]   PulsarSourceOrderedE2ECase>SourceTestSuiteBase.testSourceSingleSplit:105->SourceTestSuiteBa se.generateAndWriteTestData:315 » BrokerPersistence
[ERROR]   PulsarSourceOrderedE2ECase>SourceTestSuiteBase.testSourceSingleSplit:105->SourceTestSuiteBa se.generateAndWriteTestData:315 » BrokerPersistence
[ERROR]   PulsarSourceOrderedE2ECase>SourceTestSuiteBase.testTaskManagerFailure:232 » BrokerPersisten ce
[ERROR]   PulsarSourceOrderedE2ECase>SourceTestSuiteBase.testTaskManagerFailure:232 » BrokerPersisten ce
[ERROR]   PulsarSourceUnorderedE2ECase>UnorderedSourceTestSuiteBase.testOneSplitWithMultipleConsumers :60 » BrokerPersistence
[ERROR]   PulsarSourceUnorderedE2ECase>UnorderedSourceTestSuiteBase.testOneSplitWithMultipleConsumers :60 » BrokerPersistence

2022-01-20T15:28:37.1467261Z Jan 20 15:28:37 [ERROR] org.apache.flink.tests.util.pulsar.PulsarSourceUnorderedE2ECase.testOneSplitWithMultipleConsumers(TestEnvironment, ExternalContext)[2]  Time elapsed: 77.698 s  <<< ERROR!
2022-01-20T15:28:37.1469146Z Jan 20 15:28:37 org.apache.pulsar.client.api.PulsarClientException$BrokerPersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available
2022-01-20T15:28:37.1470062Z Jan 20 15:28:37    at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:985)
2022-01-20T15:28:37.1470802Z Jan 20 15:28:37    at org.apache.pulsar.client.impl.ProducerBuilderImpl.create(ProducerBuilderImpl.java:95)
2022-01-20T15:28:37.1471598Z Jan 20 15:28:37    at org.apache.flink.connector.pulsar.testutils.runtime.PulsarRuntimeOperator.sendMessages(PulsarRuntimeOperator.java:172)
2022-01-20T15:28:37.1472451Z Jan 20 15:28:37    at org.apache.flink.connector.pulsar.testutils.runtime.PulsarRuntimeOperator.sendMessages(PulsarRuntimeOperator.java:167)
2022-01-20T15:28:37.1473307Z Jan 20 15:28:37    at org.apache.flink.connector.pulsar.testutils.PulsarPartitionDataWriter.writeRecords(PulsarPartitionDataWriter.java:41)
2022-01-20T15:28:37.1474209Z Jan 20 15:28:37    at org.apache.flink.tests.util.pulsar.common.UnorderedSourceTestSuiteBase.testOneSplitWithMultipleConsumers(UnorderedSourceTestSuiteBase.java:60)
2022-01-20T15:28:37.1474949Z Jan 20 15:28:37    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022-01-20T15:28:37.1475658Z Jan 20 15:28:37    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2022-01-20T15:28:37.1476383Z Jan 20 15:28:37    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-01-20T15:28:37.1477030Z Jan 20 15:28:37    at java.lang.reflect.Method.invoke(Method.java:498)
2022-01-20T15:28:37.1477670Z Jan 20 15:28:37    at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
2022-01-20T15:28:37.1478388Z Jan 20 15:28:37    at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)

Issue Links

• is blocked by: FLINK-26182 E2E test for PulsarSink on new Flink sink testing tools
• is blocked by: FLINK-27199 Bump Pulsar to 2.10.0 for fixing the unstable Pulsar test environment.
• is fixed by: FLINK-27199 Bump Pulsar to 2.10.0 for fixing the unstable Pulsar test environment.
• is related to: FLINK-26237 PulsarSourceOrderedE2ECase failed on azure due to timeout
• relates to: FLINK-23944 PulsarSourceITCase.testTaskManagerFailure is instable