Details
-
Bug
-
Status: Open
-
Blocker
-
Resolution: Unresolved
-
cdc-3.1.1
-
None
-
None
-
jdk 17
flink 1.18
flinkcdc 3.0.0
Description
I integrated Flink with SpringBoot to complete my project because I only used a small part of Flink's functionality, namely the data extraction feature in Flink CDC. I don't want to build another Flink environment to manage and maintain, I want to integrate this feature with other businesses and manage it through Spring. I can use the Idea development tool to package and start projects normally. I use the spring boot man plugin tool to package, but when I start it using Java Jar and execute StreamExecutionEnvironment. Execute(), it prompts that the class cannot be found. However, this class does exist and can be packaged
org.apache.flink.runtime.client.JobInitializationException: Could not start the JobMaster. at org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1773) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: java.util.concurrent.CompletionException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.flink.api.common.ExecutionConfig at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320) at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1770) ... 3 more Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.flink.api.common.ExecutionConfig at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321) at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:114) at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768) ... 3 more Caused by: java.lang.ClassNotFoundException: org.apache.flink.api.common.ExecutionConfig at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:467) at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78) at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2045) at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1909) at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2235) at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1744) at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:514) at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:472) at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:539) at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:527) at org.apache.flink.util.SerializedValue.deserializeValue(SerializedValue.java:67) at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:101) at org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:122) at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:379) at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:356) at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:128) at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:100) at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112) ... 4 more
I think the root cause lies in this class loader
It was found that when running locally and packaging with springboot, the classloader for the same class is different. The locally launched classlaoder is sun.misc Launcher$AppClassLoader。
But after executing the Java jar startup, the class loader is org.springframework.boot.loader LaunchedURLClassLoader， This is because the Spring project has customized the classloader to load files
After packaging, the class loader of StreamExecutioneEnvironment is loaded by springboot's class loader, but we call the method to pass getSystemclassloader, -->sun.misc Launcher$AppClassLoader， According to the parental delegation mechanism, upward search is definitely not possible
So I use the constructor of StreamExecutionEnvironment to pass the classloader when calling, passing in the current thread's classloader. As shown in the figure below, the current thread's classloader is Tomcat Embedded Webapp ClassLoader, and the parent classloader is org. springframework. boot. loader LaunchedURLClassLoader, StreamExecutioneEnvironment is created by org.springframework.boot.loader Launched by URLClassLoader
But after packaging, there is no effect. I don't know how to solve this problem. Do you have any good ideas?
