[FLINK-31324] Broken SingleThreadFetcherManager constructor API

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.17.0, 1.18.0
• Component/s: Connectors / Parent
• Labels: pull-request-available
• Assignee: Yun Tang
• Reporter: Yun Tang
• Created: 04/Mar/23 15:40
• Updated: 06/Mar/23 10:13
• Resolved: 06/Mar/23 10:13

Description

FLINK-28853 changed the default constructor of SingleThreadFetcherManager. Though the SingleThreadFetcherManager is annotated as Internal, it actually acts as some-degree public API, which is widely used in many connector projects:
flink-cdc-connector, flink-connector-mongodb and so on.

Once flink-1.17 is released, all these existing connectors are broken and cannot be used in new release version, and will throw exceptions like:

java.lang.NoSuchMethodError: org.apache.flink.connector.base.source.reader.fetcher.SingleThreadFetcherManager.<init>(Lorg/apache/flink/connector/base/source/reader/synchronization/FutureCompletingBlockingQueue;Ljava/util/function/Supplier;)V
	at com.ververica.cdc.connectors.mysql.source.reader.MySqlSourceReader.<init>(MySqlSourceReader.java:91) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
	at com.ververica.cdc.connectors.mysql.source.MySqlSource.createReader(MySqlSource.java:159) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
	at org.apache.flink.streaming.api.operators.SourceOperator.initReader(SourceOperator.java:312) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask.init(SourceOperatorStreamTask.java:94) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:699) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:675) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:921) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
	at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_362]

Thus, I suggest to make the original SingleThreadFetcherManager constructor as depreacted instead of removing it.

Issue Links

• is caused by: FLINK-28853 FLIP-217 Support watermark alignment of source splits
• links to: GitHub Pull Request #22094