Details
-
Bug
-
Status: Closed
-
Blocker
-
Resolution: Fixed
-
1.14.4, 1.15.0, 1.16.0
Description
The Pulsar connector didn't use the consuming position from the checkpoint. They just commit the position to Pulsar after the checkpoint is complete. And the connector starts to consume messages from Pulsar directly by the offset stored on the Pulsar subscription.
This causes the test could be failed in some situations. The start cursor (position on Pulsar) would be reset to the wrong position, which caused the results didn't match the desired records.
How to fix this issue
Change the start position seeking mechanism from Pulsar consumer API to Pulsar admin API. Don't reset the start position when the topic has a subscription.
This issue fixes
Attachments
Issue Links
- blocks
-
FLINK-26721 PulsarSourceITCase.testSavepoint failed on azure pipeline
-
- Closed
-
-
FLINK-23944 PulsarSourceITCase.testTaskManagerFailure is instable
-
- Closed
-
-
FLINK-24872 PulsarSourceITCase.testMultipleSplits failed on AZP
-
- Closed
-
-
FLINK-25815 PulsarSourceITCase.testTaskManagerFailure failed on azure due to timeout
-
- Closed
-
-
FLINK-26177 PulsarSourceITCase.testScaleDown fails with timeout
-
- Closed
-
-
FLINK-27833 PulsarSourceITCase.testTaskManagerFailure failed with AssertionError
-
- Closed
-
-
FLINK-25884 PulsarSourceITCase.testTaskManagerFailure failed on azure
-
- Closed
-
- is related to
-
FLINK-28972 Add methods of StartCursor and StopCursor to align the Java
-
- Closed
-
- links to