[IMPALA-11342] Hive UDFs are unable to use classes from the same Jar in catalogd - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: Impala 4.1.0
Fix Version/s: Impala 4.2.0, Impala 4.1.1
Component/s: Catalog
Labels:
None

Epic Color:
ghx-label-14

Description

While it is a known limitations that Hive UDFs in Impala cannot load dependencies from classpath, they should be able to load classes from the same jar (and thus creating an shaded jar with the dependencies should be a valid workaround). Meanwhile I got an error when the parent class of the UDF was a class defined in another package in the jar.

The exception occurred when catalogd tried to instantiate the UDF class:

I0608 14:50:21.520296 22464 jni-util.cc:286] java.lang.NoClassDefFoundError: com/esri/core/geometry/Point
        at java.lang.Class.getDeclaredConstructors0(Native Method)
        at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
        at java.lang.Class.getConstructor0(Class.java:3075)
        at java.lang.Class.getConstructor(Class.java:1825)
        at org.apache.impala.hive.executor.HiveLegacyJavaFunction.instantiateUDFInstance(HiveLegacyJavaFunction.java:115)
        at org.apache.impala.hive.executor.HiveLegacyJavaFunction.<init>(HiveLegacyJavaFunction.java:74)
        at org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:50)
        at org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:67)
        at org.apache.impala.service.CatalogOpExecutor.createFunction(CatalogOpExecutor.java:2117)
        at org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:453)
        at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:245)
Caused by: java.lang.ClassNotFoundException: com.esri.core.geometry.Point
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 11 more