[FLINK-22632] CatalogFunctionImpl.isGeneric should use ContextClassLoader

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.12.2, 1.12.3
• Fix Version/s: 1.13.0
• Component/s: Connectors / Hive
• Labels: easyfix
• Assignee: Unassigned
• Reporter: slankka
• Created: 11/May/21 11:53
• Updated: 12/May/21 04:29
• Resolved: 11/May/21 14:02

Description

Hello, community.

I'm using Hive catalog, I'm trying to load UDF through HTTP.

I registered a UDF, then I submited a FlinkSQL job with command line:

flink run ... -C "http://someHost/plusTwoFunc.jar" ...

When the job started, it throws an Exception:

Caused by: java.lang.ClassNotFoundException: com.slankka.flink.udf.PlusTwoFunc
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.flink.table.catalog.CatalogFunctionImpl.isGeneric(CatalogFunctionImpl.java:72)

I know that user code is loaded by SafetyNetWrapperClassLoader . and FlinkUserCodeClassLoader , SafetyNetWrapperClassLoader is a facade classloader, FlinkUserCodeClassLoader is the real classloader whose classpath contains the jar dependency, but it failed to load udf in CatalogFunctionImpl

@Override
public boolean isGeneric() {
    if (functionLanguage == FunctionLanguage.PYTHON) {
        return true;
    }
    try {
        Class c = Class.forName(className); //ClassNotFound
        if (UserDefinedFunction.class.isAssignableFrom(c)) {
            return true;
        }
    } catch (ClassNotFoundException e) {
        throw new RuntimeException(String.format("Can't resolve udf class %s", className), e);
    }
    return false;
}

 When I change to:

Thread.currentThread().getContextClassLoader().loadClass(className);

Then it works fine.

My proposal is CatalogFunctionImpl.isGeneric should using ContextClassloader, that is FlinkUserCodeClassLoader proxied by SafetyNetWrapperClassLoader

Issue Links

• is fixed by: FLINK-20606 sql cli with hive catalog cannot create function using user classes from jar which specified by -j option