[FLINK-29890] UDFs classloading from JARs in 1.16 is broken

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.16.0
• Component/s: Table SQL / Runtime
• Assignee: Unassigned
• Reporter: Alexander Fedulov
• Created: 04/Nov/22 14:48
• Updated: 24/Feb/23 09:55

Description

1.16 introduced a lot of changes with respect to classloading in the Table API. The way UDFs could previously be loaded from JARs in 1.15 does not work in 1.16 anymore - it fails with the ClassNotFound exception when UDFs are used at runtime. 

Here is a repository with a reproducible example:
https://github.com/afedulov/udfs-flink-1.16/blob/main/src/test/java/com/example/UDFTest.java
 
It works as is (Flink 1.15.2) and fails when switching the dependencies to 1.16.0.

Here are some of the PRs that might be related to the issue:
https://github.com/apache/flink/pull/20001
https://github.com/apache/flink/pull/19845
https://github.com/apache/flink/pull/20211 (fixes a similar issue introduced after classloading changes in 1.16)
 
It is unclear how UDFs can be loaded from JARs in 1.16.
Ideally, this should be covered by tests and described in the documentation.